{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "_cell_guid": "0b9ffbb2-8d33-eb0c-5a42-0733bdc3b74b",
    "_uuid": "e14f65723f5ac826104e861c45e58525740d8b2a"
   },
   "source": [
    "# Bike Sharing 数据集上的回归分析\n",
    "\n",
    "1、 任务描述 请在Capital Bikeshare （美国Washington, D.C.的一个共享单车公司）提供的自行车数据上进行回归分析。训练数据为2011年的数据，要求预测2012年每天的单车共享数量。\n",
    "\n",
    "原始数据集地址：http://archive.ics.uci.edu/ml/datasets/Bike+Sharing+Dataset 1) 文件说明 \n",
    "day.csv: 按天计的单车共享次数（作业只需使用该文件） \n",
    "hour.csv: 按小时计的单车共享次数（无需理会） \n",
    "readme：数据说明文件\n",
    "\n",
    "2) 字段说明 \n",
    "Instant记录号 \n",
    "Dteday：日期 \n",
    "Season：季节（1=春天、2=夏天、3=秋天、4=冬天） \n",
    "yr：年份，(0: 2011, 1:2012) \n",
    "mnth：月份( 1 to 12) \n",
    "hr：小时 (0 to 23) （只在hour.csv有，作业忽略此字段） \n",
    "holiday：是否是节假日 \n",
    "weekday：星期中的哪天，取值为0～6 \n",
    "workingday：是否工作日 1=工作日 （是否为工作日，1为工作日，0为非周末或节假日 weathersit：天气（1：晴天，多云 ",
    "2：雾天，阴天 ",
    "3：小雪，小雨 ",
    "4：大雨，大雪，大雾） temp：气温摄氏度 \n",
    "atemp：体感温度 \n",
    "hum：湿度 \n",
    "windspeed：风速 \n",
    "\n",
    "casual：非注册用户个数 \n",
    "registered：注册用户个数 \n",
    "cnt：给定日期（天）时间（每小时）总租车人数，响应变量y\n",
    "casual、registered和cnt三个特征均为要预测的y，作业里只需对cnt进行预测"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {
    "_cell_guid": "2ba3154a-c2aa-3158-1984-63ad2c0c786a",
    "_uuid": "5eb696b95780825e94ddb49787f9fa339fc3833b"
   },
   "outputs": [],
   "source": [
    "# 导入必要的工具包\n",
    "# 数据读取及基本处理\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "\n",
    "#模型\n",
    "from sklearn.linear_model import LinearRegression, RidgeCV, LassoCV, ElasticNetCV\n",
    "\n",
    "#模型评估\n",
    "from sklearn.metrics import mean_squared_error\n",
    "from sklearn.metrics import r2_score  #评价回归预测模型的性能\n",
    "\n",
    "#可视化\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {
    "_cell_guid": "21fa35be-878b-b4f2-ef6e-68dc070b8bfa",
    "_uuid": "73aee228226be55c0c8a6e4fcbf818c56cd94926"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>instant</th>\n",
       "      <th>season_1</th>\n",
       "      <th>season_2</th>\n",
       "      <th>season_3</th>\n",
       "      <th>season_4</th>\n",
       "      <th>mnth_1</th>\n",
       "      <th>mnth_2</th>\n",
       "      <th>mnth_3</th>\n",
       "      <th>mnth_4</th>\n",
       "      <th>mnth_5</th>\n",
       "      <th>...</th>\n",
       "      <th>weekday_5</th>\n",
       "      <th>weekday_6</th>\n",
       "      <th>temp</th>\n",
       "      <th>atemp</th>\n",
       "      <th>hum</th>\n",
       "      <th>windspeed</th>\n",
       "      <th>holiday</th>\n",
       "      <th>workingday</th>\n",
       "      <th>yr</th>\n",
       "      <th>cnt</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0.355170</td>\n",
       "      <td>0.373517</td>\n",
       "      <td>0.828620</td>\n",
       "      <td>0.284606</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>985</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.379232</td>\n",
       "      <td>0.360541</td>\n",
       "      <td>0.715771</td>\n",
       "      <td>0.466215</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>801</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.171000</td>\n",
       "      <td>0.144830</td>\n",
       "      <td>0.449638</td>\n",
       "      <td>0.465740</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1349</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.175530</td>\n",
       "      <td>0.174649</td>\n",
       "      <td>0.607131</td>\n",
       "      <td>0.284297</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1562</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.209120</td>\n",
       "      <td>0.197158</td>\n",
       "      <td>0.449313</td>\n",
       "      <td>0.339143</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1600</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 35 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   instant  season_1  season_2  season_3  season_4  mnth_1  mnth_2  mnth_3  \\\n",
       "0        1         1         0         0         0       1       0       0   \n",
       "1        2         1         0         0         0       1       0       0   \n",
       "2        3         1         0         0         0       1       0       0   \n",
       "3        4         1         0         0         0       1       0       0   \n",
       "4        5         1         0         0         0       1       0       0   \n",
       "\n",
       "   mnth_4  mnth_5  ...   weekday_5  weekday_6      temp     atemp       hum  \\\n",
       "0       0       0  ...           0          1  0.355170  0.373517  0.828620   \n",
       "1       0       0  ...           0          0  0.379232  0.360541  0.715771   \n",
       "2       0       0  ...           0          0  0.171000  0.144830  0.449638   \n",
       "3       0       0  ...           0          0  0.175530  0.174649  0.607131   \n",
       "4       0       0  ...           0          0  0.209120  0.197158  0.449313   \n",
       "\n",
       "   windspeed  holiday  workingday  yr   cnt  \n",
       "0   0.284606        0           0   0   985  \n",
       "1   0.466215        0           0   0   801  \n",
       "2   0.465740        0           1   0  1349  \n",
       "3   0.284297        0           1   0  1562  \n",
       "4   0.339143        0           1   0  1600  \n",
       "\n",
       "[5 rows x 35 columns]"
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 读入数据\n",
    "data = pd.read_csv(\"FE_day.csv\")\n",
    "data.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_cell_guid": "a1ae46d1-8787-67a3-2f69-6497c97320eb",
    "_uuid": "16fa6600f9877c9607060f9e50de6eaa9bc1c766"
   },
   "source": [
    "**准备训练数据**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "F:\\deeplearn\\lib\\site-packages\\sklearn\\utils\\validation.py:475: DataConversionWarning: Data with input dtype int64 was converted to float64 by MinMaxScaler.\n",
      "  warnings.warn(msg, DataConversionWarning)\n"
     ]
    }
   ],
   "source": [
    "# 采用最小最大缩放 MinMaxScaler 去量纲\n",
    "from sklearn.preprocessing import MinMaxScaler\n",
    "\n",
    "# 从原始数据中分离输入特征X和输出y\n",
    "y = data['cnt']\n",
    "X = data.drop(['cnt'], axis = 1)\n",
    "\n",
    "feat_names = X.columns\n",
    "\n",
    "# 分别初始化对特征和目标的标准化器\n",
    "ss_X = MinMaxScaler()\n",
    "ss_y = MinMaxScaler()\n",
    "\n",
    "# 分别对训练和测试数据的特征以及目标值进行标准化处理\n",
    "# 对训练数据，先调用fit方法训练模型，得到模型参数,然后对训练数据和测试数据进行transform\n",
    "X = ss_X.fit_transform(X)\n",
    "y = ss_y.fit_transform(y.values.reshape(-1, 1))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 保存特征工程的结果到文件，供机器学习模型使用\n",
    "fe_data = pd.DataFrame(data = X, columns = feat_names, index = data.index)\n",
    "fe_data['cnt'] = y\n",
    "\n",
    "fe_data.to_csv('FE_bikesharing.csv', index=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "# get labels\n",
    "data = pd.read_csv('FE_bikesharing.csv')\n",
    "y = data['cnt']   \n",
    "X = data.drop(['cnt'], axis=1)\n",
    "feat_names = X.columns"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "train samples: (584, 34)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "F:\\deeplearn\\lib\\site-packages\\sklearn\\model_selection\\_split.py:2026: FutureWarning: From version 0.21, test_size will always complement train_size unless both are specified.\n",
      "  FutureWarning)\n"
     ]
    }
   ],
   "source": [
    "# 用train_test_split 分割训练数据和测试数据\n",
    "from sklearn.model_selection import train_test_split\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, train_size = 0.8,random_state = 0)\n",
    "\n",
    "print(\"train samples:\" ,X_train.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "F:\\deeplearn\\lib\\site-packages\\pandas\\core\\frame.py:3697: SettingWithCopyWarning: \n",
      "A value is trying to be set on a copy of a slice from a DataFrame\n",
      "\n",
      "See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/indexing.html#indexing-view-versus-copy\n",
      "  errors=errors)\n"
     ]
    }
   ],
   "source": [
    "# #保存测试ID，用于结果提交\n",
    "# testID = X_test['instant']\n",
    "\n",
    "# #ID不参与预测\n",
    "# X_train.drop(['instant'], axis=1, inplace = True)\n",
    "# X_test.drop(['instant'], axis=1, inplace = True)\n",
    "\n",
    "# #保存特征名字以备后用（可视化）\n",
    "# feat_names = X_train.columns"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_cell_guid": "9fe1c63e-3803-feac-362c-631399fdb8ec",
    "_uuid": "431749cca6d8fbbc2d1d3732baeb6d63590c6e2e"
   },
   "source": [
    "**1* Linear Regression without regularization**\n",
    "最小二乘线性回归\n",
    "最小二乘没有超参数需要调优，直接用全体训练数据训练模型"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {
    "_cell_guid": "101cf15a-9006-ac9a-8abe-756371fd8b1a",
    "_uuid": "38562579d6306877a067189f25ce15ececbe99d2",
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "RMSE on Training set : 0.0881617316583601\n",
      "RMSE on Test set : 0.09024008478018512\n",
      "r2_score on Training set : 0.8377853512135114\n",
      "r2_score on Test set : 0.8553466627498196\n"
     ]
    }
   ],
   "source": [
    "# Linear Regression\n",
    "# 1. 生成学习器实例\n",
    "lr = LinearRegression()\n",
    "\n",
    "#2. 在训练集上训练学习器\n",
    "lr.fit(X_train, y_train)\n",
    "\n",
    "#3.用训练好的学习器对训练集/测试集进行预测\n",
    "y_train_pred = lr.predict(X_train)\n",
    "y_test_pred = lr.predict(X_test)\n",
    "\n",
    "\n",
    "rmse_train = np.sqrt(mean_squared_error(y_train,y_train_pred))\n",
    "rmse_test = np.sqrt(mean_squared_error(y_test,y_test_pred))\n",
    "print(\"RMSE on Training set :\", rmse_train)\n",
    "print(\"RMSE on Test set :\", rmse_test)\n",
    "\n",
    "r2_score_train = r2_score(y_train,y_train_pred)\n",
    "r2_score_test = r2_score(y_test,y_test_pred)\n",
    "print(\"r2_score on Training set :\", r2_score_train)\n",
    "print(\"r2_score on Test set :\", r2_score_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Ridge picked 34 features and eliminated the other 0 features\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAacAAAEVCAYAAABE7SrmAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzt3XmcHFW9/vHPwxogERCiBgNEWUQhEiTsyw2BCyiLG4jgkqCgeOGH/ly4uYoat3tRFLgiiIAim8giYgAFlRCSYBIJkIXIEpZgIhECBgQxLOF7/zinSduZpbunp6tm5nm/XvOiu+pU1ame0N+pqlNPKSIwMzMrkzWK7oCZmVktFyczMysdFyczMysdFyczMysdFyczMysdFyczMysdFyfrlyS9RdLdkp6VdLKk9SRdL+kZSVdL+pCk39axni9KurAdfe6iD1tIek7Smi1a30RJl7ViXf1BI5+HpCmSjuvtPpmLkxVM0jGSZucv36WSfiNp7xas+hRgSkQMiYjvA0cArwc2iYgjI+LyiDiwu5VExH9HRI+/jCSNkBSS1mp02Yj4c0QMjoiVTWx3jKQljS7X4Db2lDQ5/yHwTP4j4G319EHScEm/kPRkXna+pPGdtB2TP8Nra6bvmKdPaeV+WbFcnKwwkj4LnAX8N6lwbAGcC7y7BavfElhQ8/6BiHi5Beu2TNIewG+BXwGbAW8C5gK3S3pzHau4FFhM+v1sAnwUeLyL9suAPSVtUjVtHPBA4723UosI//in7T/AhsBzwJFdtFmXVLweyz9nAetWzT8UmAM8DfwBeHuePhlYCazI27gCeBF4Kb//ODAemF61ru2B3wF/I305fjFPnwhcVtVu97ytp0lfwmOq5k0BvgHcDjxL+tLeNM/7MxB5+88BewBbA7cBzwBPAld28jmMyMuu1d12apbbAPgn8ErVdjfL+3QVcElefgEwumq5zYBfkArBI8DJXfyOpgHndjD9N8Al+fUYYEknyz8HjKrz38wYYAlwHnBinrZmnvYV0pFype2ewB35s70D2LNq3pvy5/5s/p3/oMHf8XFF//8zEH4K74B/BuYPcDDwcuULt5M2XwdmAq8DhuYvjG/kee8AngB2y19Q44BF5OJV+yXC6kVmPLk4AUOApcDngEH5/W61ywFvBJ4C3kU66/Dv+f3Qqm0+BGwLrJffn5bnjaCqwORpVwBfyusaBOzdyefwL8t2tZ0Oll2tMOR9WpH3Y03gf4CZed4awJ35y34d4M3Aw8BBHax7fdIfAft1MO9YYGlnfahq93tSkf0gsEU3/2bGkArRnsCsPO1dwM3AceTiBLwWWA58BFgLODq/3yTPnwGcQfrjZ19SkWrkd+zi1IYfn9azomwCPBldn2b7EPD1iHgiIpYBXyN94QAcD/woImZFxMqIuBh4gfRXb6MOBf4aEd+LiBUR8WxEzOqg3YeBX0fEryPilYj4HTCb9EVWcVFEPBAR/yQdnYzqYrsvkU5nbZa3O72BPjeynY5Mz/uxknRqbcc8fRfSF/HXI+LFiHgYuIBUPGq9lvQFvrSDeUuBTevox5Gko68vA49ImiNpl64WiIg/AK+V9BbSacBLapocAiyMiEsj4uWIuAK4DzhM0hZ5H78cES9ExFTg+qpl6/kdWxu4OFlRngI27WaAwGbAo1XvH83TIH2pf07S05UfYPOq+Y3YnHQk0p0tgSNrtrk3MKyqzV+rXj8PDO5ifacAAv4oaYGkjzXQ50a2U8/yg/LvYktgs5p9/CLpmmCt5aRThsM6mDeMdKqySxGxPCImRMT2eRtzgOskqZtFLwVOAvYDflkzr/bfDfn9G/O85RHxj5p5FfX8jq0NGh45ZNYiM0inlt4DXNNJm8f414ENW+RpkC6ifysivtWCviwmnfqpp92lEXF8E9tYLf4/Iv5KOgIkj1D8vaSpEfFgE+uve7vdWAw8EhHbdLviiH9ImkE6+rm1ZvYHgFsa2XBEPCnpu6RTtK8l/QHTmUuBB0nXtZ6vqWWVfzfVtgBuIh3RbSxpg6oCtQWrPqee/I6thXzkZIWIiGdI1zXOkfQeSetLWlvSOyV9Jze7AjhV0lBJm+b2lftRLgBOkLSbkg0kHSJpSBPduQF4g6TPSFpX0hBJu3XQ7jLSqaGDJK0paVAe3jy8jm0sIx1lvDqCTdKRVcsuJ31BNjxcvBuPA5tI2rDO9n8E/i7pP/O9YWtK2qGLU20TgHH5XrIhkjaW9E3SgI+vVTfMn1f1jyR9O69/rfy7+xTwYER0VZiIiEeAfyNds6v1a2DbfJvCWpKOAt4G3BARj5JO031N0jr5j4LDqpbtye/YWsjFyQoTEWcAnwVOJX15LyadqrkuN/km6YtkHjAfuCtPIyJmk446fkD6Yn+QNMihmX48S7rwfRjpdNdC0umi2naLScPcv1jV3y9Qx/9HEfE88C3SEOunJe1OuvYxS9JzwCTg0/lLt2Ui4j5SkX84b7fL0575GtRhpGtYj5BOzV1IGl3ZUfvpwEHA+0hHJY8CO5EGdyysavpG0sjB6p+tSIMqfkkaGfcw6Yjn8Dr3bXpEPNbB9KdI1xE/Rzr6OgU4NCIqpxmPIQ2k+RvwVaquWfXkd2ytpQg/bNDMzMrFfw2YmVnpuDiZmVnpuDiZmVnpuDiZmVnp+D6nJm266aYxYsSIorthZtan3HnnnU9GxNDu2rk4NWnEiBHMnj276G6YmfUpkmrTOzrk03pmZlY6PnKyPuWWyVsV3QWzAW//sfVEUfaMj5xqSJok6Z6i+2FmNpC5OFWR9D7Sw8/MzKxAbStOOZjzRklzJd0j6ShJO0u6TdKdkm6WNCy3PV7SHbntLyStn6cfmZedK2lqnjZI0kWS5ku6W9J+efp4SddKuknSwqow0c76N5iU8/bNLtp8QtJsSbOXLVvWqo/GzMxqtPPI6WDgsYjYMSJ2IMXXnw0cERE7Az8hBWMCXBsRu0TEjsC9pMdqQ0qlPihPr4RDnggQESNJjz24WNKgPG8UcBQwEjhK0uZd9O8bwPdIz7bpUEScHxGjI2L00KHdjoQ0M7MmtbM4zQcOyBH5+5Ae8LYD8DtJc0jJ1JVY+h0kTZM0n/Q01O3z9NuBn0o6nvR4aUgPArsUXk1gfpT0+GqAWyLimYhYAfyJ1Z/xAoCkUcDWEVH70DIzMytA20brRcQDknYmPe74f4DfAQsiYo8Omv8UeE9EzJU0HhiT13FCfs7OIcCcXFS6emLmC1WvV9L5/u4B7CxpUW7zOklTImJMfXtn7dKOUUJmVrx2XnPaDHg+Ii4Dvkt6nspQSXvk+WtLqhwhDQGWSlqbdORUWcdWETErIr5Ces7M5sDUShtJ25Keanl/I32LiB9GxGYRMYJ0JPaAC5OZWXHaeZ/TSOB0Sa8AL5GeePky8P38lM61gLNIj+T+MjCLdIpuPqlYkZffhnS0dAswF7gPOC+fAnwZGB8RL0hdHVCZmVmZ+WGDTRo9enQ4vsjMrDGS7oyI0d21831OZmZWOgMuvkjSLGDdmsnHAxOBrUgDJ66PiAlt7prVYeLEiUV3wWzAa8f/hwOuOEXEbrXT8k2+342IWyWtA9wi6Z0R8Zv299DMzJwQAUTE8xFxa379InAXq+65MjOzNnNCRA1JGwGHkUYD1s5zfJGZWRs4IaKKpLWAK4DvR8TDtfMdX2Rm1h5OiPhX5wMLI+Ks7vfIiuABEWYDgxMiVq37m8CGwGea3EUzM2sRJ0QAkoYDX8rruisv+4OIuLBHe2xmZk1xQkSTnBBhZtY4J0SYmVmfNeBuwu0kIeIjETG/iP6YmdnqBlxx6ighAkDSTcAw0mcyDTgxIla2s2/WvSUTphXdBbMBb/hp+/T6Nnxab5UP5Jt7dwCGAkcW3B8zswHL8UVZRPw9v1wLWAfwSBEzs4I4vqiKpJuBJ4BngWs6mO/4IjOzNnB8UZWIOIh03WldYGwH8x1fZGbWBo4vWr2fKyRNAt6d+2gl0o4LsWZWPMcXpeUGV13vWotUQO9remfNzKxHHF+UbABMkrQu6XThZOC8HuyrmZn1gOOLmuT4IjOzxjm+yMzM+qwBlxDh+CIzs/IbcMWpi/iibwEfBTaOiMHt7ZXV63tHHVp0F8wGvM9deUOvb8On9Va5Hti16E6YmZnji14VETMjYmk3++CECDOzNnB8UQOcEGFm1h7tvOY0H/iupG8DNwDLWRVfBOn+osqRyw6SvglsBAwGbs7TK/FFVwHX5ml7k4ocEXGfpNXiiwAkVeKLFvfaHpqZWUs4vsj6lHZciDWz4jm+yMzMSsfxRVkeMHEMsL6kJcCFETGx+d01M7NmOb6oSY4vMjNrnOOLzMyszxpwAwQcX2RmVn6lK06SpgCfj4hOz5nlEXyjI+KkRtffUXyRpBMk/Yw0ou854BMR8adG122975wTJhfdBeunTjxvtYdfW4F8Wi/5WUSMjIhRwHeAM4rukJnZQNbj4iTpFEkn59dnSpqcX+8v6TJJB0qaIekuSVdLGpzndxhdVLXeNSRdnG/GRdKxkh6QdBuwV1W7wyTNytFFv5f0+rzsQklDq9b1oKRNO9qHiPh71dsNgA5HiTi+yMysPVpx5DQV2Ce/Hg0Mzvcn7U0aBn4qcEBEvAOYDXw2z+8sugjS6cbLgQci4tRcuL5GKkr/Drytqu10YPeI2An4OXBKRLwCXMaqe6QOAOZGxJOd7YSkEyU9RDpyOrmjNo4vMjNrj1Zcc7oT2FnSEFIiw12kIrUPMIlUSG7P9x2tA8wA3kLn0UUAPwKuiohKwdoNmBIRywAkXcmqiKLhwJW5gK0DPJKn/wT4FeneqY8BF3W1ExFxDnCOpGNIBXVcox+EmZm1Ro+LU0S8JGkRcCzwB2AesB+wFalQ/C4ijq5eRtJIOo8uIq9nP0nfi4gVlU110vZs4IyImCRpDDAx92uxpMcljSUVtw91snytnwM/rLOttZkvWpsNDK0aEDEV+Hz+7zTgBGAOMBPYS9LWAJLWzxFD99N5dBHAj4FfA1dLWouUFjFG0ib5lOCRVW03BP6SX9ce7VxIOr13VUSs7KzzOXWi4hBgYd17bmZmLdeq4jQNGAbMiIjHgRXAtHwabjxwhaR5pGK1XUS8CBwBfFvSXFIh27N6hRFxBukU4aXA46QjohnA7/P0iomkIjaNlLdXbRIp1bzLU3rASZIWSJoDfBaf0jMzK1S/ji+SNBo4MyL26bZxgxxfZGbWuHrji0p3E26rSJpACpet91qTmZmVRL8tThFxGnBa9TRJX+Jfr1cBXF01KtBK7t7t3lp0F6yfeut99xbdBatSuoQISVPy6biu2oyX9ING1x0R34qIUTU/rxYmSUdIiu62b2Zmvat0xako+T6tk0kjA83MrECOL1rlG6R0iBWdNXB8kZlZezi+CJC0E7B5RNzQ1Y46vsjMrD0GfHyRpDWAM0n3Y5mZWQk4vgiGkArllFwo3wBMknR4V8+UsmJ4RJXZwDDg44si4pmI2DQiRkTEiNxnFyYzswI5vsjMzErH8UVNcnyRmVnjHF/k+CIzsz6r3xYnxxf1TyMvHll0F6yfmj9uftFdsCqlS4goIr5I0r75JuGXJR3RfO/NzKwVSlecCvJn0sCNnxXcDzMzw/FFAETEooiYB7zSzb46vsjMrA0cX9QAxxeZmbXHgI8vMjOz8nF8kfUpHlFlNjAM+PgiMzMrH8cXAZJ2kbSEVPR+JGlBIztvZmat5fiiJjm+yMyscY4vcnyRmVmf1W+Lk+OL+qmJGxbdA+uvJj5TdA+sSukSIgqKL/qspD9JmifpFklbNr8HZmbWU6UrTgW5GxgdEW8HrgG+U3B/zMwGNMcXARFxa0Q8n9/OJN3Y29G+Or7IzKwNHF+0uo8Dv+lohuOLzMzaw/FFVSR9OPf93xr6BMzMrKUcX7SqTwcAXwL+LSJe6KqtFcgjqswGBMcXpe3vRDpaOzwinmh4783MrKUcX5ScnttdLWmOpEl177mZmbWc44ua5PgiM7PGOb7I8UVmZn1Wvy1Oji/qn0ZMuLHoLlg/tei0Q4ruglUpXUJEQfFF4yUty9eb5kg6rvk9MDOznuq3R05NuDIiTiq6E2Zm5viiRvfV8UVmZm3g+KJV3q+USn6NpM07auD4IjOz9nB8UXI9cEVEvCDpBOBiYGzDn4T1Ol+0NhsYenzkFBEvAYtYFV80jdXjiyqDD94WER8HRIovqkwfGREHVq22El80qHpTnXThbOAHETES+CQwKPdrMVAdX9RhmGtu+1RVZNEFwM4NfARmZtZiji9K26++3nU4cG/de25mZi3n+KLkZEkLcl9Ozn02M7OCOL6oSY4vMjNrnOOLHF9kZtZnla44SZoCfD4iOj0skTQeGN3VTbONxhdJ+gDpFGGQhp0f09QOWK9yfFHreOSjlVnpilNvykPTV8vRk7QN8F/AXhGxXNLr2t45MzN7lRMikuOBcyJiOYAfOGhmViwnRCTbAttKul3STEkHd9TI8UVmZu3hhIhkLWAbYExe3zRJO0TE09WNIuJ84HxIo/W6/2jMzKwZPS5OEfGSpEWsSoiYx+oJEUdXLyNpJCkhYo9OVltJiPheRKyobKqTtmcDZ0TEJEljSIMaiIjFkqoTIroatbcEmJnTLh6RdD+pWN3RxTJWAF/ENxsYnBCRXEcqqOTrUtsCDzew/2Zm1kJOiEhuBp6S9CfgVuALEfFUvTtvZmat5YSIJjkhwsyscU6IcEKEmVmf1W+LU6MJEW3rmJmZdavfFCdJGwHHRMS5+f0YUgzSoZU2nSVE5PbTgCH57euAP0bEe3q10wPEG26d07J1/XW/US1bl5mVV6sGRJTBRsB/NLtwROxTefghaeDFtS3rmZmZNaRUxUnSCEn3SbpQ0j2SLpd0QE5uWChpV0kTJf1E0hRJD1eik0in8LaSNEfS6XnaYEnX5HVernzHbzd9GEJ6RPt1vbSbZmbWjTKe1tuadF3oE6SbYI8hRSEdDnyRNOx8O9J9SUOA+yX9EJgA7JCPfCqn9XYCtgceA24nxR9N72b77wVuiYi/186Q9IncL7bYYoue7KOZmXWhVEdO2SMRMT/n4y0gFYog5fSNyG1ujIgXclbeE8DrO1nXHyNiSV7XnKrlu3I0cEVHMyLi/IgYHRGjhw4dWv8emZlZQ8p45PRC1etXqt6/wqr+VrdZSef7UW87ACRtAuxKOnqyFvEgBjNrVBmPnJr1LKtG2zXrSOCGqjw/MzMrQL8pTjlu6PY8kOL0bhfo2Afp5JSemZm1T7+OL+pNji8yM2tcvfFF/ebIyczM+o8yDojoVZJ+CbypZvJ/RsTNRfTHzMxW12+KUz3xRQAR0eVIPElnA8dGxODe6utAc8vkrVq2rv3HPtSydZlZefWn03o9ii+CVx+xsVFrumNmZs0qVXEqMr5I0prA6cApXbT5hKTZkmYvW7asZfttZmb/qlTFKdsa+F/g7aSYokp80edJ8UXk6QeRbpj9an50+wTgoRze+oXcbifgM8DbgDeT4os6cxIwKSKWdtbACRFmZu1RxuLU9vgiSZuRbsA9u3W7YWZmzSrjgIgi4ot2Ih2xPZjP/K0v6cGI2LqBflsnPIjBzBpVxuLUrKbjiyLiRuANlfeSnnNhMjMrThlP6zWlRfFFZmZWAo4vapLji8zMGuf4IjMz67P60zWnuji+yMys/PpNceppfJGkHwOjAQEPAOMj4rle7fQAMXHixFKuy8zKqz+d1utpfNH/j4gdI+LtwJ9JN+WamVkBSlWciowvioi/5z4IWA9YbaSI44vMzNqjVMUpKyq+CEkXAX/N618tLcLxRWZm7VHG4tT2+KKKiDgW2Ay4Fziqx3tiZmZNKeOAiCLii14VESslXQl8Abiong5b1zyIwcwaVcYjp2Y1HV+kZOvKa+Aw4L4W9s3MzBpQxiOnpkTEU3ngxD3Ab4AbG1hcwMWSXpNfzwU+1QvdNDOzOji+qEmOLzIza5zji8zMrM/qN6f16uX4ovZbMmFay9Y1/LR9WrYuMyuvflOcWhBfdDkpvugl4I/AJyPipV7ttJmZdag/ndbraXzR5aSbb0eSEiKOa0WnzMyscaUqTgXHF/06MtKR0/AO+uf4IjOzNihVccoKiy8CyOv6CHBT7TzHF5mZtUcZi1Nh8UXZucDUiGjdVXwzM2tIGQdEFBZfJOmrwFDgk/V21rrnEXZm1qgyFqdmNR1fBCDpONKpwv3zkZaZmRWkjKf1mhIRTwG354EUp3e7wOrOI50enJEHVXyltT00M7N6Ob6oSY4vMjNrnOOLzMysz+pP15zq4vii9vveUYd236hOn7vyhpaty8zKq98cOUnaSNJ/VL0fI2m1b7KIeG++F6r652ZJJ0l6UFJI2rS9vTczs2r9pjjR8/ii24EDgEdb0x0zM2tWqYpTwfFFd0fEom765/giM7M2KFVxygqNL+qK44vMzNqjjMWp6PgiMzMrWBlH6xUWX2S9wyPszKxRZTxyalaP4ovMzKw8+k1x6ml8kaSTJS0hPcdpnqQLW95JMzOri+OLmuT4IjOzxjm+yMzM+qzCBwhIWgSMziPvqqf/ISL2bPU2+lp80TknTC66C6Vy4nlji+6CmbVBocVJ0pqdzWtVYepgve/tjfWamVnrNH1aT9IplXQGSWdKmpxf7y/pMklHS5qfByh8u2q55yR9XdIsYI+q6etJuknS8ZV2+b9jchrEakkPkt6Vp02X9P1Klp6kTST9VtLdkn4EqGo710m6U9ICSZ/I0z4u6cyqNsdLOqPZz8bMzHqmJ9ecpgKV52+PJkUFrU1Kc1gIfBsYC4wCdpH0ntx2A+CeiNgtIqbnaYOB64GfRcQFHWxrtaQHSYOAHwHvjIi9SY9Xr/gqMD0idgImAVtUzftYROyc+3yypE2AnwOH5/4DHAtcVNsJxxeZmbVHT4rTncDOkoaQbnadQfrC3wd4GpgSEcsi4mXgcmDfvNxK4Bc16/oVcFFEXNLJtjpKetgOeDgiHsltrqhqvy9wGUBE3Agsr5p3sqS5wExgc2CbiPgHMBk4VNJ2wNoRMb+2E44vMjNrj6aLU0S8BCwiHWX8AZgG7AdsBfy5i0VXRMTKmmm3A+/sIpi1o6SHTkNcK12snSBpDCl5fI+I2BG4GxiUZ18IjKeToyYzM2ufng6ImEoKZP0YKfvuDNIR1UzgrPxcpOXA0cDZXaznK8CXgXOBT9W57fuAN0sakdPEj6rp14eAb0p6J7Bxnr4hsDwins9HSLtXFoiIWZI2B95BCp0tBY9OM7OBqKf3OU0DhgEzIuJxYAUwLSKWAv8F3ArMBe6KiF91s67PAIMkfaeeDUfEP0nPb7pJ0nTgceCZPPtrwL6S7gIOZNWR3E3AWpLmAd8gFdFqVwG3R8RyzMysMH06IULS4Ih4Lp8OPAdYGBFndrdcF+u7ATgzIm7prq0TIszMGjdQEiKOlzSH9GiNDUmj9xqWH/H+APDPegqTmZn1rsITInoiHyU1faRUtZ6ngW173iMzM2uF0hUnSSOAGyJihzrb/zS3vyYniZ8REX+qaTOeFF90Umt72/vu3e6tRXehVN56371Fd8HM2qB0xaknIuK4ovtgZmY9V9ZrTmtKuiBHDP02RxuNkjRT0jxJv5S0ce1COeZodH59rKQHJN0G7FXV5jBJs3K00e8lvV7SGpIWShqa26wh6cE8FN7MzNqsrMVpG+CciNielDbxfuASUnr420n3VH21s4UlDSMNJ98L+HdS7FHFdGD3HG30c+CUnDxxGeneKEg36s7tICnd8UVmZm1Q1uL0SETMya/vJKVObBQRt+VpF7MqDqkju7EqPulF4MqqecOBmyXNB74AbJ+n/wT4aH79MTpIiXB8kZlZe5T1mlNtXNFGTayjsxu4ziYNmpiU44wmAkTEYkmPSxpLKm4f6mT5tvIAADMbiMp65FTrGWC5pEoK+keA27poPwsYkx+dsTZwZNW8DYG/5Nfjapa7kHR676oO8v/MzKxNynrk1JFxwHmS1gceJgW0digilkqaSEpKXwrcBVQebDgRuFrSX0jxRdVPxZ1EOp3n4FczswL16fiiVssj/c6MiH26a+v4IjOzxtUbX9SXjpx6laQJpET0UlxrMjMbyPrKNadeFxGnRcSWVU/nNTOzgrT9yClHCf02Ih7L7xeRooWe7Gq5Jrbza+CY/PaYiDi3m/Y3kZ7vND0iDm1lXzoy8uKRvb2Jfmn+uNUeUGxm/VARR07jgc1asSJJnRbXiHhXDnTdiPTcp+6cThoFaGZmBeu2OEk6RdLJ+fWZkibn1/tLukzSgZJmSLpL0tWSBuf5X5F0h6R7JJ2v5AhgNHC5pDmS1sub+X95+fn5CbVI2kDST/I67pb07jx9fN7O9cBvJQ2TNDWv757KcHNJi3L80GnAVnn+6Z3tZ35UxrNNfo5mZtZC9Rw5TQUqo9dGA4PzvUN7k2KETgUOiIh3ALOBz+a2P4iIXXK6+HrAoRFxTW7zoYgYlZ9mC/BkXv6HpMe+A3wJmBwRuwD7AadL2iDP2wMYFxFjSafubo6IUcCOQCVZomIC8FDe3hfq+VA64/giM7P2qKc43QnsLGkIKblhBqlI7QP8k5Rbd3t+6N84YMu83H45YHU+MJZVMUEdubZqWyPy6wOBCXm9U4BBwBZ53u8i4m/59R3Asfm+ppER0WtHP44vMjNrj24HRETES3nQwrHAH4B5pCOZrYBHSIXi6OplJA0CziUNdFicC8egLjZTiStaWdUnAe+PiPtr1r0b8I+q/k2VtC9wCHCppNMj4pLu9qtovrBvZta5egdETCWdbpsKTANOIJ0+mwnsJWlrAEnrS9qWVYXoyXwN6oiqdT0LDKljmzeTrkUpr3unjhpJ2hJ4IiIuAH4MvKOmSb3bMzOzkqi3OE0DhgEzIuJxYAUwLSKWkUbfXSFpHqlYbZdHyV1AuiZ1HenUW8VPSTFE1QMiOvINYG1gnqR78vuOjAHmSLqb9GiN/62eGRFPkU473tPVgAhJ04Crgf0lLZF0UBd9MzOzXuT4oiY5vsjMrHH1xhc5IcLMzEpnQGXrSRoJXFoz+YWI2K2I/piZWccGVHxRRMwHRnVVVSi2AAAGEElEQVTQdhTpHqvXkEYMfisirqxt11ITN+zV1fdbE58pugdm1gaOL0qeBz4aEdsDBwNnSWrm6btmZtYCji8CIuKBiFiYXz8GPAGsdpetEyLMzNrD8UU1JO0KrAM8VDvPCRFmZu1RzzWn2viiu1gVXzSJVfFFkL7UZ+Tl9pN0CrA+8FpgAXB9J9uoji96X359IHC4pEqx6iq+6Ce5YF4XEbXFqW6ShpEGTIyLiFeaXY+ZmfWM44tWrfc1wI3AqRExs9HlG+YL+2ZmnXJ8UVrHOsAvgUsi4uo6+mZmZr3I8UXJB4B9gfG5X3Py8HIzMyuA44ua5PgiM7PGOb7IzMz6LMcXOb7IzKx0HF/Eq4MqrgXWJF3nOjsizmtlf2qNmHBjb66+31p02iFFd8HM2sDxRclSYM98I+9upMfDt6SPZmbWOMcXARHxYkRU7rVat7PPxfFFZmbt4fiiTNLmeTj8YuDbldOO1RxfZGbWHvUUp9r4ohmsii/6J6vii+YA44At83L7SZolaT4wFti+i21UxxeNyK8PJJ1emwNMoev4omNzCsXIiHi2jn1aTUQsjoi3A1sD4yS9vpn1mJlZzzm+qEZEPCZpAan4XtPserrjC/tmZp1zfFFax/DK9S9JGwN7Afd3tYyZmfUexxclbwVmSZoL3AZ8Nw87NzOzAji+qEmSlgGPFt2PBm0KtPR+sgL0h32A/rEf3ody6Gv7sGVEdDuizMVpAJE0u55MqzLrD/sA/WM/vA/l0B/2oSOOL3J8kZlZ6Qyo4tRZfJGZmZWLU8kHlvOL7kAL9Id9gP6xH96HcugP+7AaX3MyM7PS8ZGTmZmVjouTmZmVjovTACPpSEkLJL0iqU8NP5V0sKT7JT0oaULR/WlGTtp/It9Y3ufkgORbJd2b/x19uug+NUPSIEl/lDQ378fXiu5TsyStmZ/ccEPRfWklF6eB5x7gfaQoqj5D0prAOcA7SWHDR0t6W7G9aspPgYOL7kQPvAx8LiLeCuwOnNhHfw8vAGMjYkfSCN6DJe1ecJ+a9Wng3qI70WouTgNMRNxbG6bbR+wKPBgRD0fEi8DPgXcX3KeGRcRU4G/dNiypiFgaEXfl18+SvhTfWGyvGhfJc/nt2vmnz40OkzScFHp9YdF9aTUXJ+sr3kh61lbFEvrgl2J/ImkEsBMwq9ieNCefDpsDPEF6ukJf3I+zgFOAV4ruSKu5OPVDkn6fg25rf/rckUYVdTCtz/2l21/kpw38AvhMRPy96P40IyJW5oeUDgd2lbRD0X1qhKRDSU9kuLPovvSGAZUQMVBExAFF96EXLAE2r3o/HFjtacXW+/KTsH8BXB4R13bXvuwi4mlJU0jXAvvSQJW9gMMlvYv0mKLXSLosIj5ccL9awkdO1lfcAWwj6U2S1gE+CEwquE8DTn6+2o+BeyPijKL70yxJQyVtlF+vBxwA3FdsrxoTEf8VEcMjYgTp/4fJ/aUwgYvTgCPpvZKWAHsAN0q6ueg+1SMiXgZOIj2E8l7gqohYUGyvGifpCmAG8BZJSyR9vOg+NWgv4CPA2PxMtjn5L/e+Zhhwa34O3R2ka079aih2X+f4IjMzKx0fOZmZWem4OJmZWem4OJmZWem4OJmZWem4OJmZ2asaCSeWtK+kuyS9LOmIqumjJM3IobrzJB3VaD9cnMzMrNpPqT+c+M/AeOBnNdOfBz4aEdvndZ1Vua+sXk6IMDOzV0XE1Jyb+CpJW5GeCjCUVHiOj4j7ImJRnv9KzToeqHr9mKQn8rJP19sPFyczM+vO+cAJEbFQ0m7AucDYehaUtCuwDvBQIxt0cTIzs07lkN89gatTehUA69a57DDgUmBcRDSUnO7iZGZmXVkDeDonuNdN0muAG4FTI2JmMxs1MzPrUH4kyiOSjoQU/itpx66WyeHMvwQuiYirm9mus/XMzOxVOZx4DLAp8DjwVWAy8ENSYO7awM8j4uuSdiEVoY2BFcBfI2J7SR8GLgKqw5nHR8Scuvvh4mRmZmXj03pmZlY6Lk5mZlY6Lk5mZlY6Lk5mZlY6Lk5mZlY6Lk5mZlY6Lk5mZlY6/wdhpBiZFVVprAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Plot important coefficients\n",
    "coefs = pd.Series(lr.coef_, index = feat_names)\n",
    "print(\"Ridge picked \" + str(sum(coefs != 0)) + \" features and eliminated the other \" +  \\\n",
    "      str(sum(coefs == 0)) + \" features\")\n",
    "\n",
    "#正系数值最大的10个特征和负系数值最小（绝对值大）的10个特征\n",
    "imp_coefs = pd.concat([coefs.sort_values().head(10),\n",
    "                     coefs.sort_values().tail(10)])\n",
    "imp_coefs.plot(kind = \"barh\")\n",
    "plt.title(\"Coefficients in the OLS Model\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "instant        -5.461446e-01\n",
       "season_1        4.579730e+12\n",
       "season_2        4.579730e+12\n",
       "season_3        4.579730e+12\n",
       "season_4        4.579730e+12\n",
       "mnth_1         -3.835704e+11\n",
       "mnth_2         -3.835704e+11\n",
       "mnth_3         -3.835704e+11\n",
       "mnth_4         -3.835704e+11\n",
       "mnth_5         -3.835704e+11\n",
       "mnth_6         -3.835704e+11\n",
       "mnth_7         -3.835704e+11\n",
       "mnth_8         -3.835704e+11\n",
       "mnth_9         -3.835704e+11\n",
       "mnth_10        -3.835704e+11\n",
       "mnth_11        -3.835704e+11\n",
       "mnth_12        -3.835704e+11\n",
       "weathersit_1   -1.595166e+12\n",
       "weathersit_2   -1.595166e+12\n",
       "weathersit_3   -1.595166e+12\n",
       "weekday_0       1.678356e+12\n",
       "weekday_1       3.064769e+12\n",
       "weekday_2       3.064769e+12\n",
       "weekday_3       3.064769e+12\n",
       "weekday_4       3.064769e+12\n",
       "weekday_5       3.064769e+12\n",
       "weekday_6       1.678356e+12\n",
       "temp            3.313293e-01\n",
       "atemp           9.994507e-02\n",
       "hum            -1.679583e-01\n",
       "windspeed      -1.436462e-01\n",
       "holiday        -1.386413e+12\n",
       "workingday     -1.386413e+12\n",
       "yr              4.990234e-01\n",
       "dtype: float64"
      ]
     },
     "execution_count": 48,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "coefs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "系数的值非常大。由于特征之间强相关，OLS模型的性能并不好"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_cell_guid": "a70803d8-638c-bdfa-6897-aed8ce78f475",
    "_uuid": "2938aee78a9f61d6ca59345d6fa0793e2422f34c"
   },
   "source": [
    "**2* Linear Regression with Ridge regularization (L2 penalty)**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Best alpha : 1.0\n",
      "cv of rmse : [0.09245886 0.09210729 0.09186617 0.09475474 0.12102217 0.19040304]\n",
      "RMSE on Training set : 0.08679376504867618\n",
      "RMSE on Test set : 0.0894323571809375\n",
      "r2_score on Training set : 0.8427803218009013\n",
      "r2_score on Test set : 0.8579246208902165\n"
     ]
    }
   ],
   "source": [
    "#RidgeCV缺省的score是mean squared errors \n",
    "# 1. 设置超参数搜索范围，生成学习器实例\n",
    "# RidgeCV(alphas=(0.1, 1.0, 10.0), fit_intercept=True, normalize=False, scoring=None, cv=None, gcv_mode=None, store_cv_values=False)\n",
    "alphas = [0.01, 0.1, 1, 10, 100, 1000]\n",
    "ridge = RidgeCV(alphas = alphas,store_cv_values=True )\n",
    "\n",
    "# 2. 用训练数据度模型进行训练\n",
    "# RidgeCV采用的是广义交叉验证（Generalized Cross-Validation），留一交叉验证（N-折交叉验证）的一种有效实现方式\n",
    "ridge.fit(X_train, y_train)\n",
    "\n",
    "#通过交叉验证得到的最佳超参数alpha\n",
    "alpha = ridge.alpha_\n",
    "print(\"Best alpha :\", alpha)\n",
    "\n",
    "# 交叉验证估计的测试误差\n",
    "mse_cv = np.mean(ridge.cv_values_, axis = 0)\n",
    "rmse_cv = np.sqrt(mse_cv)\n",
    "print(\"cv of rmse :\",rmse_cv)\n",
    "\n",
    "#训练上测试，训练误差，实际任务中这一步不需要\n",
    "y_train_pred = ridge.predict(X_train)\n",
    "rmse_train = np.sqrt(mean_squared_error(y_train,y_train_pred))\n",
    "\n",
    "y_test_pred = ridge.predict(X_test)\n",
    "rmse_test = np.sqrt(mean_squared_error(y_test,y_test_pred))\n",
    "\n",
    "print(\"RMSE on Training set :\", rmse_train)\n",
    "print(\"RMSE on Test set :\" ,rmse_test)\n",
    "\n",
    "r2_score_train = r2_score(y_train,y_train_pred)\n",
    "r2_score_test = r2_score(y_test,y_test_pred)\n",
    "print(\"r2_score on Training set :\" ,r2_score_train)\n",
    "print(\"r2_score on Test set :\" ,r2_score_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {
    "_cell_guid": "1fb3f0d6-a070-9ce5-7cc3-01a7e26faa4f",
    "_uuid": "659084128a0ef75a5936912a5f64134774b29664",
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Ridge picked 34 features and eliminated the other 0 features\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAacAAAEICAYAAAD7pTujAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzt3XucVVX9//HXO1Px9pUUKs0LiRapKCpqphZe0lLzkvr1QiVWlplZv/KWmVFmWdTXLDNT805mXvOWSBiCiCgqMKDmDcwLKWh4SSOVz++PtSYOhzMz55w5cy4z7+fjMY/ZZ++11/7sPTBr1tprf7YiAjMzs2byjkYHYGZmVsyNk5mZNR03TmZm1nTcOJmZWdNx42RmZk3HjZOZmTUdN07W0iR9UNKDkl6VdJykVSTdJOllSVdLGinp9jLqOUXShfWIuZMYNpD0mqQValTfaElX1KKuMo7V6XWWNFHSF+sRSy1UEq+kkLRxT8fU17hxsrqQdLik6fmX73xJf5a0Uw2qPhGYGBFrRMQvgYOA9wBrR8TBETE2IvboqpKI+FFEdPuXp6RB+ZfVOyvdNyL+HhGrR8TbVRx3hKRnKt2vgvovkfSf/PN7SdJ4SUPat5d7nXsgrtH5eh9XtP4bef3oesdkteHGyXqcpG8CvwB+RGo4NgDOBfarQfUbAnOKPj8aEW/VoG5b1k8jYnXgfcCzwO8aHE+7R4EjitZ9Lq+3FuXGyXqUpDWBHwBfjYjrIuJfEfFmRNwUESfkMitL+oWk5/LXLyStXFDHPpJmSFok6W5JW+T1dwC7AOfkv+ivBE4DDsmfvyBplKS7CuraLP/V/5Kk5yWdktcvMwQm6cP5WIskzZQ0omDbREmnS5qShxNvlzQgb56Uvy/KMewgaWNJd+ahxoWSrurgWi3T6+riOIX7rQb8GVg3H/M1SevmzStJuizvP0fS8IL91pV0raQFkuYW9z46EhFvAH8EhhXUVXydPy7pkXzO5wAq2LaCpJ/nazFX0rFF572mpN/lHvazkn7YxVDnfcCqkjbL+28GrJLXF16noyQ9nn/2NxZco07jzds/L+lhSf+UNE7ShuVcK6ueGyfraTsA/YDrOynzHeDDpF92WwLbAacCSNoauAj4MrA28FvgRkkrR8SuwGTg2Dwcdhipd3ZV/rzMX/aS1gD+AtwGrAtsDEwoDkbS+4BbgB8CawHHA9dKGlhQ7HDgSODdwEq5DMBH8/f+OYapwOnA7cC7gPWAX3VyLYp1dJz/ioh/AZ8EnsvHXD0insub9wX+APQHbgTOyef4DuAmYCapJ7Qb8A1Je3YVUG4MDwMe72D7AOBa0s9wAPAEsGNBkaNyvMOArYH9i6q4FHiL9PPZCtgD6GrI9XJSbwlSL+qyoph2BX4M/C+wDvAU6bp0Ga+k/YFTgE8DA0n/5q7sIh7rJjdO1tPWBhZ2Mcw2EvhBRLwQEQuA7wOfzduOAn4bEdMi4u2IuBRYTGrMKrUP8I+I+HlE/DsiXo2IaSXKfQa4NSJujYglETEemA7sVVDm4oh4tFQvooQ3ScON6+bj3tVJ2WKVHKeUu/J5vE36Bb5lXr8tMDAifhAR/4mIJ4ELgEM7qet4SYuAV4GdWPozKrYX8FBEXBMRb5KGdP9RsP1/gbMj4pmI+CdwZvsGSe8hNVzfyL3sF4CzuogL4ArgMEkr5rLFE0FGAhdFxAMRsRj4NrCDpEFlxPtl4McR8XD+d/wjYJh7Tz3LjZP1tBeBAep8gsC6pL9k2z2V10H6pf6tPLy2KP9yXL9geyXWJ/1V3JUNgYOLjrkT6S/udoW/vF4HVu+kvhNJw0T35qG1z1cQcyXHKWf/fvlnsSFpGLDwHE8h3RPsyM8ioj8wCHgD+GAH5dYFnm7/ECm79NMdbS9a3hBYEZhfENdvST3HDkXE30k9uR8Bj0XE00VFlvk3FhGvkf5tvq+MeDcEzi6I5yXSz/N9ncVk3VPxjCKzCk0F/k0aurmmgzLPsezEhg3yOki/JM6IiDNqEMvTpOGocspdHhFHVXGM5dL8R8Q/SD1AlGYo/kXSpIgoOSxWpUpfL/A0MDciNqn4QBF/l/R14FJJN+deXaH5pD8EAJCkws95+3oFnwu3PU3qGQ+oYlLLZaQh4CNLbGv/N9Ye02qkXv2zZcTb/m9wbIXxWDe452Q9KiJeJk1S+LWk/SWtKmlFSZ+U9NNc7ErgVEkD8/j/aSwdlrkAOFrS9kpWk7R3vn9UqZuB9ypNM15Z0hqSti9R7grgU5L2zDfv+ylN1V6vRNliC4AlwEbtKyQdXLDvP0kNScXTxbvwPLC20gSUctwLvCLpJKVnw1aQtLmkbcvZOQ91Pgd8qcTmW4DNJH0699KOA95bsP2PwNclvU9Sf+Ckgnrnk+7P/VzS/0h6h6TBkj5WRlhXke5P/bHEtt8DR0oapjTZ5kfAtIiYV0a85wHfLphwsaakg8uIx7rBjZP1uIj4P+CbpBvOC0h/iR4L3JCL/JB0T2cW0AY8kNcREdNJvY5zSL/YHwdGVRnHq8DHgU+RhrseI832Ky73NGma+ykF8Z5AGf9fIuJ14AxgSh4G+jDp/s40Sa+RJiV8PSLmVnMOnRz3EVIj/2Q+bqfDnvke1KdI97DmAguBC4FyGzeAMcCJKphZmeteCBxMupf0IrAJMKWgyAWkBmgW8CBwK2kCRHuD/TnS5I+HSD/za1h2SLWjc3ojIv5SoidHREwAvkua+DAfGEy+j9VVvBFxPfAT4A+SXgFmk+6LWQ+SXzZoZo0k6ZPAeRHhCQb2X+45mVld5WHEvSS9M0/b/x6dP2pgfZB7TmZWV5JWBe4EhpBm/d1CGup8paGBWVNx42RmZk3Hw3pmZtZ0/JxTlQYMGBCDBg1qdBhmZi3l/vvvXxgRA7sq58apSoMGDWL69OmNDsPMrKVIeqrrUh7WMzOzJtSyPaf8ZPnhEXFuo2Mxq7UJdwxudAhmHdpt13JSVHZPK/ec+gPHNDoIMzOrvVZunM4EBiu9hG6MpBMk3SdplqTvw39f3vaIpAslzZY0VtLuSi9ve0zSdrncaEmXS7ojr68m4aeZmdVIKzdOJwNPRMQwYDwpH9Z2pFxh20hqf+nbxsDZwBakh/4OJ73+4HhS7rR2WwB7k16Od1qp3GSSviRpuqTpCxYs6JmzMjOzlm6cCu2Rvx4kJQ0dQmqsIL0WoC0ilpBeyTAhv6+ljfRemnZ/yokjFwJ/JTV0y4iI8yNieEQMHziwy5mQZmZWpZadEFFEpDdV/naZlektl4sLVi0p+LyEZc+/OFWGU2eYmTVIKzdOrwLt7/QZB5wuaWxEvJaTSb5ZYX37SfoxsBowgjRsaNYQ9ZgNZdbMWrZxiogX88SG2cCfSS8Tm5peYslrwGeo7IVu95ISUG4AnB4Rz3VR3szMekjLNk4AEXF40aqzSxTbvKD8qILleYXbgEcjotRbPc3MrM56y4QIMzPrRVq651QrETG60TGYmdlSbpzMmtDo0aMbHYL1Mc32b87Deh2QtEKjYzAz66v6bM9J0unAwog4O38+A3geOACYT8o0sWnjIjQz67v6cs/pd8ARAJLeARwKPEvKDPGdiFiuYXL6IjOz+uizjVOeSv6ipK1YmvroReDeiJjbwT5OX2RmVgd9dlgvuxAYBbwXuCiv+1fDojHLmu3mtFm99dmeU3Y98AlgW1IKJDMzawJ9uucUEf+R9FdgUUS8nVMfmZlZg/XpxilPhPgwcDBAREwEJjYwJDMzow8P60naFHic9H6nxxodj5mZLdWyjZOkU7ou1bGIeCgiNoqIb9UqJjMzq41WHtY7BfhRo4Mw6wnPnDy50SFYi1rvzJ0bHUJNtETPSdINku6XNCc/CHsmsIqkGZLG5jKfkXRvXvfb9vRDkl6T9JO8/18kbSdpoqQnJe2by4yS9CdJt0n6m6TvNfB0zcz6vJZonIDPR8Q2wHDgOGAM8EZEDIuIkZI+BBwC7BgRw0gvGRyZ910NmJj3fxX4IfBxUpqiHxQcY7u8zzDgYEnD63BeZmZWQqsM6x0n6YC8vD6wSdH23YBtgPvydPBVgBfytv8At+XlNmBxRLwpqQ0YVFDH+Ih4EUDSdcBOwPTCg0j6EvAlgA022KD7Z2VmZiU1feMkaQSwO7BDRLwuaSLQr7gYcGlEfLtEFW9GROTlJcBigIhYIqnw/KNov+LPRMT5wPkAw4cPX267mZnVRtM3TsCawD9zwzSE9FwSwJuSVoyIN4EJwJ8knRURL0haC1gjIp6q4Dgfz/u9AewPfL6WJ2FWid5yU9usWq1wz+k24J2SZgGnA/fk9ecDsySNjYiHgFOB23O58cA6FR7nLuByYAZwbURM76K8mZn1EC0d8eq7JI0ChkfEseXuM3z48Jg+3e2XmVklJN0fEV1OOGuFnpOZmfUxrXDPqcdFxCXAJQ0Ow8zMMveczMys6dS955Tv79weEc/lz/NI93sW1vg4twKH54+HR8S5XZS/jTQT8K6I2KeWsZhV6ueH+J9gu29ddXOjQ7AGaETPaRSwbi0qKnpOaRkRsVdELAL6A8eUUd0Y4LO1iMvMzLqny8ZJ0omSjsvLZ0m6Iy/vJukKSXtImirpAUlXS1o9bz9N0n2SZks6X8lBpBREY3MOvFXyYb6W92/LzzIhaTVJF+U6HpS0X14/Kh/nJtLU8XUkTcr1zZa0cy43T9IA4ExgcN4+pqPzjIgJpPRGnV2LL0maLmn6ggULurp0ZmZWpXJ6TpOA9icChwOrS1qRlN6njfR80e4RsTUp3c83c9lzImLbiNiclE5on4i4JpcZmfPivZHLLsz7/wY4Pq/7DnBHRGwL7AKMkbRa3rYDcERE7EoauhuXc+ptSXpOqdDJwBP5eCeUc1E6EhHnR8TwiBg+cODA7lRlZmadKOee0/3ANpLWIKX+eYDUSO0M3AhsCkzJOe1WAqbm/XaRdCKwKrAWMAe4qYNjXFdwrE/n5T2AfSW1N1b9gPaEduMj4qW8fB9wUW4wb4iI4sbJzMxaTJeNU06SOg84ErgbmEXqyQwG5pIaisMK95HUDziXNNHhaUmjWT4fXqHF+fvbBTEJODAi/lZU9/bAvwrimyTpo8DewOWSxkTEZV2dl1kz8yQA6+vKnRAxiTTcNgmYDBxNGj67B9hR0sYAklaV9AGWNkQL8z2ogwrqehVYo4xjjiPdi1Kue6tShSRtCLwQERcAvwO2LipS7vHMzKxJlNs4TSblqpsaEc8D/wYmR8QC0uy7K3NOu3uAIXmW3AWke1I3kIbe2l0CnFc0IaKU04EVSfnzZufPpYwAZkh6EDgQOLtwY34NxpQ8WaLDCRGSJgNXA7tJekbSnp3EZmZmPci59ark3HpmZpVzbj0zM2tZfSq3nqShpNdiFFocEds3Ih4zMyutTzVOEdEGDOusjKQbgY3y81lmDfHro+9odAh18dXzdm10CNakPKxXQNKngdcaHYeZWV9Xt8YppyO6RdLMPHPuEEnbSLpT0v2SxklaJ5c9KqctminpWkmr5vUH531nSpqU1/WTdHFOffSgpF3y+lGSrpN0m6THJP20i/hWJ2W3+GEnZZy+yMysDurZc/oE8FxEbJmHzG4DfgUcFBHbABcBZ+Sy1+XUR1sCDwNfyOtPA/bM6/fN674KEBFDgcOAS/NDwJCG8A4BhgKHSFq/k/hOB34OvN5RAacvMjOrj3o2Tm3A7pJ+kpOzrg9sDoyXNIOUo2+9XHZzSZMltQEjgc3y+inAJZKOAlbI63YiT3KIiEeAp4AP5G0TIuLliPg38BCwYanAJA0DNo6I62t3umZmVq26TYiIiEclbQPsBfwYGA/MiYgdShS/BNg/ImYqvf9pRK7j6Jy+aG/Sg7fDSGmOOrK4YLkwNVKxHUj5A+flMu+WNDEiRpR3dma15YkC1tfV857TusDrEXEF8DNge2CgpB3y9hUltfeQ1gDm52SuIwvqGBwR0yLiNGAhqfc1qb1MTp20AbBMPr6uRMRvImLdiBhE6ok96obJzKxx6jmVfCjptRdLgDeBrwBvAb+UtGaO5Rek7OXfBaaRhujaWJobb4ykTUi9pQnATOARUjqktlzfqIhYnFPymZlZC3L6oio5fZGZWeWcvsjMzFpWn8oQASBpGrBy0erP5uwRZk3h4SEfanQIPepDjzzc6BCsyfWanpOk/pKOKfg8QtJyb2yLiO3zK9sLv9ok7SrpgfyQ76WS+lzDbWbWLHpN4wT0B47pslQJkt4BXAocmh8Qfgo4ooaxmZlZBZqqcZI0SNIjki7MPZixknaXNCWnINpO0mhJF0maKOlJScfl3c8EBueXGLa/VHB1SdfkOse2v1W3hLVJ2ckfzZ/Hk15cWByf0xeZmdVBUzVO2cakt9luAQwBDic9e3Q8cEouMwTYE9gO+F5+Hupk4Ik8THdCLrcV8A1gU2AjYMcOjrkQWFFS+wySg0jPUC3D6YvMzOqjGRunuRHRFhFLSM88TYg0370NGJTL3BIRiyNiIfAC8J4O6ro3Ip7Jdc0o2H8Zuf5DgbMk3Qu8SnpmyszMGqAZb/oXphxaUvB5CUvjLTctUbnliIipwM4AkvZgaX4+s7rzbDbr65qx51StV1maSaJikt6dv68MnAScV6O4zMysQr2mcYqIF4EpeSLFmC53WN4Jkh4GZgE3RUTfeBWpmVkTcvqiKjl9kZlZ5Zy+yMzMWlYzTojoUZKuB95ftPqkiBjXiHjMShl66dBGh9CltiOc8ct6Tq9pnCT1Bw6PiHPz5xHA8RGxT2G5iDigg/0vAT4GvJxXjYqIGT0WsJmZdag3DetVnb6owAkF+fbcMJmZNUhTNU4NTF9UbnxOX2RmVgdN1ThljUhf1O4MSbMknZWfd1qG0xeZmdVHMzZOdU9flH2b1OhtC6xFehDXzMwaoBknRDQqfdH89n0kXUzqqZk1hGfCWV/XjD2nanU3fdE6+buA/YHZNYrLzMwq1Iw9p6pExIt54sRs4M/ALRVWMVbSQECkIcCjax2jmZmVx+mLquT0RWZmlXP6IjMza1lNO6wn6e6I+EgV++0PPBoRD3WwvdP0RZIGAR+JiN9Xemyzmhm9ZqMjSEa/3HUZsx7QtI1TNQ1Ttj9wM1CyceoofVGBQaRnq9w4mZk1SNMO60l6LX8fkbNBLJfpQdKZkh7KD87+TNJHgH2BMTlTxGBJR0m6T9JMSddKWjXve4mkX0q6O2eaOCgf+kxg57z//2vEuZuZ9XVN23MqshWwGfAcMAXYUdJDwAHAkIgISf0jYpGkG4GbI+IaAEmLIuKCvPxD4AvAr3K965CyTwwBbgSuIWWaWC5hbN7/S8CXADbYYIMeO1kzs76uaXtORUplengF+DdwoaRPA693sO/mkiZLagNGkhq5djdExJJ8f6qjLBP/5fRFZmb10SqN03KZHiLiLVJuvWtJ95lu62DfS4BjI2Io8H2gXwf1disprJmZ1U6rDOstR9LqwKoRcauke4DH86biTBFrAPNzctiRwLNdVN2tTBNmNeFZctbHtUrPqZQ1gJslzQLuBNonL/wBOEHSg5IGA98FpgHjgUfKqHcW8FaeQOEJEWZmDeAMEVVyhggzs8o5Q4SZmbUsN05mZtZ0WnZChFlvNujkSpPq18a8M/duyHHNitW95yRplKR1Cz7PkzSgB45zq6T++euYLsoOkzRV0pycbeKQWsdjZmbla8Sw3ihg3a4KlUNSZ2+23SsiFgH9gU4bJ9IDvJ+LiM2ATwC/kNS/FjGamVnlumycJJ0o6bi8fJakO/LybpKukLRH7nU8IOnq/PwRkk7LOe1mSzpfyUHAcNKL/WZIWiUf5mt5/zZJQ/L+q0m6KNfxoKT98vpR+Tg3AbdLWkfSpFzfbEk753LtPbIzgcF5+5hS5xgRj0bEY3n5OeAFYLkUEJK+JGm6pOkLFiwo/yqbmVlFyuk5TQJ2zsvDgdXzA607AW3AqcDuEbE1MB34Zi57TkRsGxGbA6sA++R8d9OBkRExLCLeyGUX5v1/Axyf130HuCMitgV2ISVzXS1v2wE4IiJ2JWUQHxcRw4AtSemNCp0MPJGPd0JXJytpO2Al4InibU5fZGZWH+VMiLgf2EbSGqR0Pw+QGqmdSclSNwWm5EThKwFT8367SDoRWBVYC5gD3NTBMa4rONan8/IewL6S2hurfkB7ttXxEfFSXr4PuCg3mDdERHHjVDZJ6wCXkxq+JdXWY9ZdnphgfV2XjVNEvClpHnAkcDcpg8IuwGBgLqmhOKxwH0n9gHOB4RHxtKTRLJvTrlh7jru3C2IScGBE/K2o7u2BfxXEN0nSR4G9gcsljYmIy7o6r2KS/ge4BTg1Iu6pdH8zM6udcidETCINt00CJgNHk4bP7iG9vmJjAEmrSvoASxuihfke1EEFdZWbu24c6V5U+7ubtipVSNKGwAv5tRi/A7YuKtLl8SStBFwPXBYRV5cRm5mZ9aByG6fJpHcfTY2I50mvqpgcEQtIs++uzDnu7iG9X2kRcAHpntQNpKG3dpcA5xVNiCjldGBFYJak2flzKSOAGZIeBA4Ezi7cGBEvkoYdZ3c0IQL4X+CjwKgc1wxJwzqJzczMepBz61XJufXMzCrn3HpmZtayek36ovzQ7OERcW7+PIKi161LGkqajVdocURsL+l3pFmIAh4FRkXEa3UJ3qxIT6Qv8gxAayW9qefUZSaIiGjLzzsVfm2fN/+/iNgyIrYA/g4c29MBm5lZaU3VOEkaJOkRSRfmCQxjJe0uaYqkxyRtJ2l0zhwxUdKT7dkrKJ0JYnVJ1+Q6x7bP/CslIl7JMYj00LBvxpmZNUhTNU7ZxqQZd1sAQ0gZIHYiTWU/JZcZAuwJbAd8Lz+AWyoTxFbAN0gPCm8E7NjZgSVdDPwj1/+rEtudvsjMrA6asXGam4fflpCySkyINKWwDRiUy9wSEYsjYiEpD957Oqjr3oh4Jtc1o2D/kiLiSFJS2oeB5TKTO32RmVl9NOOEiMUFy0sKPi9habyFZQqzSnRWV2fl/isi3pZ0FXACcHE5AZvVmicvWF/XjD2napWbeWI5OWN6e5YLAZ8CHqlhbGZmVoFm7DlVJSJezBMnZgN/JuXJK5eAS3N+PQEzga/0QJhmZlYGZ4iokjNEmJlVzhkizMysZfWaYb1ySboeeH/R6pMiYlwj4jEzs+X1msapnPRFABFxQAf7jyWlL3oTuBf4ckS82aNBW6/03r9W/b7L//rHLk6Kb31bbxrW6zJ9URfGkh6+HUrKEPHFWgRlZmaVa6rGqcHpi26NjNRzWq+HT9fMzDrQVI1T1rD0RQC5rs8Ct5XY5vRFZmZ10IyNU8PSF2XnApMiYnLxBqcvMjOrj2acENGw9EWSvgcMBL5cbrBmxTyZwaz7mrFxqlbV6YsAJH2RNFS4W+5pmZlZgzTjsF5VIuJFYEqeSDGmyx2Wdx5peHBqnlRxWm0jNDOzcjl9UZWcvsjMrHJOX2RmZi2r6Rqn/KzT7ArKXyLpoLx8oaRNS5QZJemcvHx9HrYr/NqzdmdgZmbd1ZsmRBARXWZ16Ch9kVktTLhjcE3q2W3XJ2pSj1mrarqeU7aCpAskzZF0u6RVJA2TdI+kWbn3867inXLWiOF5+UhJj0q6k4KHbyV9StI0SQ9K+ouk90h6R85AMTCXeYekxyUNqNsZm5nZfzVr47QJ8OuI2AxYBBwIXEbKHr4F6YHc73W0s6R1gO+TGqWPkzJEtLsL+HBEbAX8ATgxTx2/AhiZy+wOzMwP+RbW6wwRZmZ10KyN09yIaE/tfD8wGOgfEXfmdZcCH+1k/+2BiRGxICL+A1xVsG09YJykNuAEYLO8/iLgc3n588DFxZU6Q4SZWX00a+NUnNmhfxV1dDRH/lfAORExlJQJoh9ARDwNPC9pV1Lj9ucqjmlmZjXQKhMiXgb+KWnnnPPus8CdnZSfBpwtaW3gFeBgYGbetibwbF4+omi/C0nDe5dHxNu1Ct76Dk9kMKuNVmmcIDUk50laFXgSOLKjghExX9JoYCowH3gAWCFvHg1cLelZ4B6WfSvujaThvOWG9MzMrH6cIaJAnul3VkTs3FVZZ4gwM6tcuRkiWqnn1KMknQx8haUz9szMrEGadUJE3UXEmRGxYUTc1ehYzMz6OjdOZmbWdHrNsJ6k/sDhEXFu/jwCOD4i9ilz/2NJr3QfDAwsfgDX+rbRo0f36uOZNZve1HPqDxzTjf2nkDJDPFWbcMzMrFpN1TjljOSP5OzisyWNlbS7pCk59912kkZLuijn0XtS0nF59zOBwTnLePvLBleXdE2uc6wkdXTsiHgwIuZ1EZ/TF5mZ1UFTNU7ZxsDZwBbAEOBwYCfgeOCUXGYI6ZXq2wHfk7QicDLwREQMi4gTcrmtSEN1mwIbUZAAthpOX2RmVh/N2DjNjYi2nIx1DjAh0sNYbcCgXOaWiFic7wu9QHq9ein3RsQzua4ZBfubmVkTa8YJEYV59ZYUfF7C0niLc+91dB7lljPrlCcomNVXM/acqvUqsEajgzAzs+7rNY1TRLwITMkTKcZ0uUMRScdJeob0So1Zki6seZBmZlYW59arknPrmZlVrtzcer2m52RmZr1Hn5sgIOl6ln1NBqTXv49rRDzWGp45eXJdj7femV0mxjfr1XpN41Ru+qKIOKCD/SezdELFu0nT0PfvuYjNzKwjvWlYr1vpiyJi5/wA7zDSSwqvq1lkZmZWkaZqnBqZvqgghjWAXYEbSmxz+iIzszpoqsYpa3T6ogNIWSleKd7g9EVmZvXRjI1To9MXHQZc2Z0TMDOz7mnGCRENS18kaW1Sb6zkpAnruzx7zqy+mrHnVK1apC86GLg5Iv5dg3jMzKxKvaZx6m76ouxQPKRnZtZwTl9UJacvMjOrnNMXmZlZy2rGCRE9yumL+rafH7JP14WawLeuurnRIZg1VEv1nCT1l3RMwecRksr+XyzpWGAosCWwe0FGiAGSZuWvuyVtWfvozcysXC3VONHNFEXAFGB34Kmi9XOBj0XEFsDpwPndOIaZmXVT3RunRqYoiogHI2JeifV3R8Q/88d7SC8cLBW70xeZmdVBo3pOjU5R1JkvAH8utcHpi8w39YVDAAAOmklEQVTM6qNRjVOjUxSVJGkXUuN0UrV1mJlZ9zVqtl7DUhR1RNIWwIXAJ/MDvdYLeRacWWtotQkRtUhRtBxJG5De3/TZiHi01vWbmVllWqpx6m6KIknHSXqGNOFhlqQL86bTgLWBc/NkC6d+MDNrIKcvqpLTF5mZVc7pi8zMrGX1yvRFTlFkxX599B2NDqEiXz1v10aHYNZQLdVzKjd9UUQc0J6aqOBrnKRjJT0uKSQNKKhniKSpkhZLOr5e52NmZqW1VONEz6Uvegk4DvhZN+o2M7MacfqitP6FiLgPeLOL2J2+yMysDpy+qAJOX2RmVh9OX2RmZk3H6YusT/DsN7PW0moTInokfZGZmTWXlmqceip9kaT35vXfBE6V9Iyk/6lp8GZmVjanL6qS0xeZmVXO6YvMzKxl9crJA9WkL5J0BvA54F0RsXpPxtfXPTzkQ40Ooel96JGHGx2CWUP1ysYpIg6oYrebgHOAx2ocjpmZVahuw3qSVpN0i6SZeULDIZK2kXSnpPsljZO0Ti57lKT7ctlrJa2a1x+c950paVJe10/SxZLaJD2YX7WOpFGSrpN0W8488dPO4ouIeyJifk9fBzMz61o97zl9AnguIraMiM2B24BfAQdFxDbARcAZuex1EbFtRGwJPAx8Ia8/Ddgzr983r/sqQEQMBQ4DLpXUL28bBhwCDAUOkbR+d07A6YvMzOqjno1TG7C7pJ9I2hlYH9gcGC9pBnAqaYo3wOaSJktqA0YCm+X1U4BLJB0FrJDX7QRcDhARj5CSun4gb5sQES9HxL+Bh4ANu3MCTl9kZlYfdbvnFBGPStoG2Av4MTAemBMRO5Qofgmwf0TMlDQKGJHrOFrS9sDewAxJw4AOE73i7BFNyTf7zawr9bzntC7wekRcQXo1xfbAQEk75O0rSmrvIa0BzM/JXkcW1DE4IqZFxGnAQlLva1J7GUkfADYA/lan0zIzsx5Qz57EUGCMpCWkV1N8BXgL+KWkNXMsvyAlgv0uMI00RNfG0pRFYyRtQuotTQBmAo8A5+UhwLeAURGxuJM3Z5SUJ0wcDqyas0VcGBGjqz9dMzOrljNEVMkZIszMKucMEWZm1rL63AQBSdOAlYtWfzYi2hoRj5mZLa/ujVOefXd7RDyXP88DhueXCtbyOLeS7iEBHB4R5wJExPYlym4o6X7S9PQVgV9FxHm1jKdVDL10aKNDMKDtCP+tZH1bI4b1RgHr1qIiSR02rhGxV0QsAvoDx3RR1XzgIxExjDSL8OQ8u9DMzBqgy8ZJ0omSjsvLZ0m6Iy/vJukKSXtImirpAUlXS1o9bz8tpyCaLel8JQcBw4GxkmZIWiUf5mt5/zZJQ/L+q0m6KNfxoKT98vpR+Tg3AbdLWkfSpFzf7PyAL5LmSRoAnAkMzttLvgMqIv4TEe3PRK1cznUxM7OeU84v4UnAznl5OLB6fv5oJ9I071OB3SNia2A66YV9AOfkFESbA6sA+0TENbnMyIgYFhFv5LIL8/6/AY7P674D3BER2wK7kKaRr5a37QAcERG7kobuxuVez5bAjKL4TwaeyMc7oaOTlLS+pFnA08BP2ocdi8o4fZGZWR2U0zjdD2wjaQ1SxoWppEZqZ+ANYFPS22lnAEewNEXQLpKm5eePdmVpCqJSris41qC8vAdpeG0GMBHoR3rAFmB8RLyUl+8DjpQ0GhgaEa+WcU7LiYinI2ILYGPgCEnvKVHG6YvMzOqgywkREfFmnrRwJHA3MIvUkxkMzCU1FIcV7pMTr55LmujwdG44+tGx9iG1whRDAg6MiGWyPeT0Rf8qiG+SpI+SUhpdLmlMRFzW1Xl1JCKekzSH1PheU209rco34s2sGZR7b2USabhtEjAZOJo0fHYPsKOkjQEkrZpTCLU3RAvzPaiDCup6laUZHzozjnQvSrnurUoVkrQh8EJEXAD8Dti6qEiXx5O0Xvv9L0nvAnbEKZDMzBqm3MZpMrAOMDUingf+DUyOiAWk2XdX5vs19wBD8iy5C0j3pG4gDb21u4SUbqhwQkQpp5Omdc+SNDt/LmUEKQnsg8CBwNmFGyPiRdKw4+yOJkQAHwKmSZoJ3An8zM89mZk1jtMXVcnpi8zMKuf0RWZm1rLq+cqMWyX1r6D8oDycV8sYhubhxMKvaSXKvVbL45qZWWXq+bLBvep1rE5iaCO9ur31jV6z0RFYTxr9cqMjMGuomvWcysgkMU/SgNwjeljSBZLmSLq9YKbcNpJmSpoKfLWg7s0k3Zt7OrMkbZLreUTSpXndNZJWLajnTkn3SxonaZ28frCk2/L6yQXZKN6vlOXiPkkdTbwwM7M6qeWwXmeZJCYXld0E+HVEbAYsIs2yA7gYOK7Eq9uPBs7OWSCGA8/k9R8Ezs8Pz74CHJOP+SvgoIjYBrgIOCOXPx/4Wl5/POlZLEgz/H6Ts1H8o6MTdIYIM7P6qGXj1FkmieLGaW5EzCjYb5DS23D7R8Sdef3lBeWnAqdIOgnYsCDt0dMRMSUvX0FqCD8IbA6Mz9klTgXWy89bfQS4Oq//LWl6PKTnmq4scdxlOEOEmVl91OyeUxeZJB4uKr64YPltUu49ASXntUfE7/PEhb2BcZK+CDxZonzkeuYU974k/Q+wKPe+Sh6m0xM0M7O6qfWEiPZMEp8nPYD7f8D9ERE50UOHImKRpJcl7RQRdwEj27dJ2gh4MiJ+mZe3IDVOG0jaISKmAocBd5EyOwxsX5+H+T4QEXMkzZV0cERcnTNPbBERM4EpwKGk3tdIWoFvmJtZL1brqeQlM0lUsP+RwK/zhIg3CtYfAszOw3FDgPbceQ+TkrTOAtYi3Tf6Dyld0k9yxocZpOE8SA3PF/L6OcB+ef3Xga9Kug/wNDgzswZr2QwRkgYBN+dXctSdM0SYmVXOGSLMzKxl1e0h3FqLiHmkWXlmZtbL9JqeU0+kOzIzs8Zo2Z5Tqxt08i2NDsGa2Lwz9250CGYN1Wt6TtkKxWmRJE2UNBwgp0+al5dHSbpB0k15ivmxkr4p6UFJ90haq6FnYmbWh/W2xqmjtEgd2Rw4HNiOlOLo9YjYipSR4nPFhZ2+yMysPnpb47RcWqQuyv81Il7Nb/R9Gbgpr28rta/TF5mZ1Udva5yK0yK9E3iLpefZr5PySwo+L8H348zMGqYv/AKeB2wD3EvKHNEUfMPbzKxjva3nVMrPgK9IuhsY0OhgzMysay2bvqjRnL7IzKxy5aYvcuNUJUkLgKcaHUc2AFjY6CA64Niq06yxNWtc4NiqVe/YNoyILmeUuXHqBSRNL+cvkUZwbNVp1tiaNS5wbNVq1tj6wj0nMzNrMW6czMys6bhx6h3Ob3QAnXBs1WnW2Jo1LnBs1WrK2HzPyczMmo57TmZm1nTcOJmZWdNx49QiJK0labykx/L3d3VQ7jZJiyTdXLT+kvxqkBn5a1gTxfZ+SdPy/ldJWqkBsR2Ryzwm6YiC9RMl/a3gur27m/F8Itf3uKSTS2xfOV+Dx/M1GVSw7dt5/d8k7dmdOGoZW37R5xsF1+i8BsT2UUkPSHpL0kFF20r+bJsgrrcLrtmNtYyrzNi+KekhSbMkTZC0YcG2HrtmZYsIf7XAF/BT4OS8fDLwkw7K7QZ8Cri5aP0lwEFNGtsfgUPz8nnAV+oZG7AW8GT+/q68/K68bSIwvEaxrAA8AWwErATMBDYtKnMMcF5ePhS4Ki9vmsuvDLw/17NCDa9Td2IbBMzuiX9bFcQ2CNgCuKzw33lnP9tGxpW3vdbga7YLsGpe/krBz7PHrlklX+45tY79gEvz8qXA/qUKRcQE4NV6BZVVHZskAbsC13S1fw/GticwPiJeioh/AuOBT9QwhnbbAY9HxJMR8R/gDzm+juK9BtgtX6P9gD9ExOKImAs8nutrhth6WpexRcS8iJhFeqNAoZ782XYnrp5WTmx/jYjX88d7gPXycr3+P3TKjVPreE9EzAfI36sZXjojd+HPkrRyk8S2NrAoIt7Kn58B3lfn2N4HPF3wuTiGi/PQy3e7+cu4q+MsUyZfk5dJ16icfbujO7EBvF/pLdJ3Stq5hnGVG1tP7NvTdfdTennpPZJq+QcZVB7bF4A/V7lvj+gLr8xoGZL+Ary3xKbv1KD6bwP/IHXxzwdOAn7QBLGV+mVf0fMNNYitsxhGRsSzktYArgU+SxqiqUY559pRmW5fpy50J7b5wAYR8aKkbYAbJG0WEa/UMbae2Len694gIp6TtBFwh6S2iHii3rFJ+gwwHPhYpfv2JDdOTSQidu9om6TnJa0TEfMlrQO8UGHd8/PiYkkXA8c3SWwLgf6S3pn/Gl8PeK7OsT0DjCj4vB7pXhMR8Wz+/qqk35OGS6ptnJ4B1i86TvG5tpd5RtI7gTWBl8rctzuqji3SjYrFABFxv6QngA8AtUrb351z7/BnWwPd+plExHP5+5OSJgJbke4T1S02SbuT/oj7WEQsLth3RNG+E2sUV9k8rNc6bgTaZ80cAfypkp3zL+b2ezz7A7ObIbb8i+2vLH0RZMXnVoPYxgF7SHpXns23BzBO0jslDQCQtCKwD927bvcBmyjNTlyJNKmgeJZWYbwHAXfka3QjcGieMfd+YBPSCzRrperYJA2UtAJA7gVsQrqJXs/YOlLyZ9vouHI8K+flAcCOwEM1iqus2CRtBfwW2DciCv9o68lrVr56z8DwV3VfpLH9CcBj+ftaef1w4MKCcpOBBcAbpL+A9szr7wDaSL9crwBWb6LYNiL9on0cuBpYuQGxfT4f/3HgyLxuNeB+YBYwBzibbs6QA/YCHiX9hfydvO4HpF8QAP3yNXg8X5ONCvb9Tt7vb8Ane+DfWFWxAQfm6zMTeAD4VANi2zb/m/oX8CIwp7OfbaPjAj6S/z/OzN+/0IBr9hfgeWBG/rqxHtes3C+nLzIzs6bjYT0zM2s6bpzMzKzpuHEyM7Om48bJzMyajhsnMzNrOm6czMys6bhxMjOzpvP/ARX8ON+SV+bzAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZIAAAEKCAYAAAA4t9PUAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzt3XmcVfV9//HXZzaGbUBgkGVAICAqi9uAS9SmEi1JjERFGUwT09rYpvXRNl2iNj+T1vpom9jl8Utj86uJSayJDIQljktE4xJjapgZdhAxI+rMMCyDwLDO/vn9cc+Nl3EYLtw5c+6d+34+HvOYs3zPuZ+DeN98z/cs5u6IiIicqZyoCxARkcymIBERkZQoSEREJCUKEhERSYmCREREUqIgERGRlChIREQkJQoSERFJiYJERERSkhd1AX1h1KhRPmnSpKjLEBHJKGvXrt3n7sWnapcVQTJp0iSqq6ujLkNEJKOY2XvJtNOpLRERSYmCREREUqIgERGRlChIREQkJQoSERFJiYJERERSoiAREZGUKEhERPqhjXUHefjlGg43t4X+WQoSEZF+6LH/fZfvvPI2uTkW+mcpSERE+pmmY208s3kXCy4ax6CC8B9goiAREelnVq2vp6W9k8VzJ/bJ5ylIRET6EXenvKqOWeOHMXP8sD75TAWJiEg/sr7uIG/uPtxnvREIOUjMbL6ZbTezGjO7t5v1A8xsabB+jZlNCpbPNbMNwc9GM7spYZt3zWxzsE6P9BURSVBeWcugglxuvGhcn31maKMwZpYLPAxcB9QDVWZW4e5vJDS7Ezjg7lPNrAz4BrAI2AKUunu7mY0FNprZU+7eHmz3u+6+L6zaRUQy0eHmNp7aGBtkHzKg794SEmaPZC5Q4+473L0VKAcWdGmzAHgsmF4OzDMzc/djCaFRCHiIdYqI9AtPbmjgeFsHZX14WgvCDZLxQF3CfH2wrNs2QXA0ASMBzOwyM9sKbAb+JCFYHHjezNaa2V0h1i8iklHKq2o5f2wRF5b0zSB7XJhB0t1dMF17Fidt4+5r3H0GMAe4z8wKg/UfdfdLgE8Af2Zm13T74WZ3mVm1mVU3Njae2RGIiGSIzfVNbNl5iMVzJ2AW/k2IicIMknpgQsJ8CdBwsjZmlgcMA/YnNnD3bcBRYGYw3xD83gusInYK7UPc/RF3L3X30uLiU75yWEQkoy2pqqUwP4cFF3U98RO+MIOkCphmZpPNrAAoAyq6tKkA7gimFwIvubsH2+QBmNk5wHTgXTMbbGZDg+WDgeuJDcyLiGStoy3tPLl+J5+aNY5hA/P7/PNDG9YPrri6G1gN5ALfd/etZvYAUO3uFcCjwONmVkOsJ1IWbH4VcK+ZtQGdwJ+6+z4zmwKsCrptecAT7v5cWMcgIpIJnt7UwNHWDm6/bMKpG4fA3Pv/BVGlpaVeXa1bTkSkf1rw8K841tLO81++plfHR8xsrbuXnqqd7mwXEclgbzQcYmPdQRbPndjng+xxChIRkQxWXlVLQV4ON1/S94PscQoSEZEMdby1g1Xrd/LJmWMYPqggsjoUJCIiGeqZzbs43Nze53eyd6UgERHJUOWVtUwZNZjLJo+ItA4FiYhIBnprz2Gq3ztAWQR3snelIBERyUDllXXk5xq3XFISdSkKEhGRTNPc1sHK9fVcP2MMI4cMiLocBYmISKZZvXU3B4+1sXhOtIPscQoSEZEM88SaWiaOGMSVHxkZdSmAgkREJKPsaDzCmnf2s2jOBHJyoh1kj1OQiIhkkPKqOvJyjFtLox9kj1OQiIhkiJb2DpavrWfe+aMZPbTw1Bv0EQWJiEiGeOGNPew/2sriiO9k70pBIiKSIcor6xg/fCBXT0uvt74qSEREMsB77x/ltZp9LJozgdw0GWSPU5CIiGSApVV15BhpNcgepyAREUlzbR2dLKuu59rzRjN22MCoy/kQBYmISJp7cdte9h1poSxN7mTvSkEiIpLmyqtqGVNUyMemp9cge5yCREQkjdUfOMYv3mrkttIS8nLT8ys7PasSEREAllXXA3DbnAkRV3JyChIRkTTV3tHJsqo6rplWTMlZg6Iu56QUJCIiaeoXbzWy+1Bz2t3J3pWCREQkTS2prGXUkAHMO3901KX0SEEiIpKGdjc189Kbe7mttIT8NB1kj0vv6kREstSy6jo6HRal8SB7XKhBYmbzzWy7mdWY2b3drB9gZkuD9WvMbFKwfK6ZbQh+NprZTcnuU0Qk03V0Okur6rhq6ijOGTk46nJOKbQgMbNc4GHgE8AFwGIzu6BLszuBA+4+FfgP4BvB8i1AqbtfBMwH/tvM8pLcp4hIRvvlbxrZefA4ZXPTvzcC4fZI5gI17r7D3VuBcmBBlzYLgMeC6eXAPDMzdz/m7u3B8kLAT2OfIiIZrbyyjpGDC7j+gjFRl5KUMINkPFCXMF8fLOu2TRAcTcBIADO7zMy2ApuBPwnWJ7NPEZGMtfdwMz/ftodbLi2hIC8zhrHDrLK7B+Z7sm3cfY27zwDmAPeZWWGS+4zt2OwuM6s2s+rGxsbTKFtEJDrL19bT3umUZcAge1yYQVIPJP5JlAANJ2tjZnnAMGB/YgN33wYcBWYmuc/4do+4e6m7lxYXp+eDzkREEnV2OuWVdVw2eQRTiodEXU7SwgySKmCamU02swKgDKjo0qYCuCOYXgi85O4ebJMHYGbnANOBd5Pcp4hIRnp9x/vU7j+W9neyd5UX1o7dvd3M7gZWA7nA9919q5k9AFS7ewXwKPC4mdUQ64mUBZtfBdxrZm1AJ/Cn7r4PoLt9hnUMIiJ96YnKWoYNzGf+zMwYZI8LLUgA3P1Z4Nkuy76WMN0M3NrNdo8Djye7TxGRTPf+kRae37qb37/8HArzc6Mu57RkxiUBIiL93Ip19bR1eMad1gIFiYhI5Nxjg+yXnnMW5549NOpyTpuCREQkYmve2c+OfUczsjcCChIRkciVV9YytDCPT80aG3UpZ0RBIiISoYPHWnl2y25uung8Awsya5A9TkEiIhKhlet20treSdmczDytBQoSEZHIuDtLKmu5cMJwLhhXFHU5Z0xBIiISkXW1B/jN3iMszqDnanVHQSIiEpEn1tQxuCCXT184LupSUqIgERGJQNPxNp7Z3MCNF41n8IBQHzISOgWJiEgEKjbspLmtk9sz9N6RRAoSEZE+5u48UVnHjHFFzCoZFnU5KVOQiIj0sU31TWzbdShj72TvSkEiItLHllTWMjA/lwUXZfYge5yCRESkDx1paadiYwOfvnAsQwvzoy6nVyhIRET6UMWGBo61dlDWT05rgYJERKRPlVfVct6YoVw8YXjUpfQaBYmISB/ZsrOJTfVNlM2ZgJlFXU6vUZCIiPSR8qpaBuTlcNPFJVGX0qsUJCIifeBYazs/Xd/Ap2aNZdig/jHIHqcgERHpA09v2sWRlnYWX9Z/BtnjFCQiIn1gSWUtU0cPofScs6IupdcpSEREQvbm7kOsrz3Y7wbZ4xQkIiIhK6+soyA3h5sv6V+D7HEKEhGREDW3dbByXT2/N3MMIwYXRF1OKBQkIiIhenbzLg41t7N4bma/BbEnChIRkRCVV9YxaeQgrpgyMupSQhNqkJjZfDPbbmY1ZnZvN+sHmNnSYP0aM5sULL/OzNaa2ebg97UJ27wS7HND8DM6zGMQETlTNXsPU/nufsrmTuyXg+xxob3f0cxygYeB64B6oMrMKtz9jYRmdwIH3H2qmZUB3wAWAfuAT7t7g5nNBFYD4xO2+6y7V4dVu4hIbyivrCMvx7ilnw6yx4XZI5kL1Lj7DndvBcqBBV3aLAAeC6aXA/PMzNx9vbs3BMu3AoVmNiDEWkVEelVLewcr1tVz/YyzKR7av7++wgyS8UBdwnw9J/YqTmjj7u1AE9D1ROItwHp3b0lY9oPgtNb9dpL+opndZWbVZlbd2NiYynGIiJy21Vv3cOBYG2Vz+t+d7F2FGSTdfcH76bQxsxnETnf9ccL6z7r7LODq4Odz3X24uz/i7qXuXlpcXHxahYuIpGrJmlpKzhrIVVNHRV1K6MIMknog8Xq3EqDhZG3MLA8YBuwP5kuAVcDn3f3t+AbuvjP4fRh4gtgpNBGRtPHOvqO8vuN9yuZMICen/w6yx4UZJFXANDObbGYFQBlQ0aVNBXBHML0QeMnd3cyGA88A97n7r+KNzSzPzEYF0/nADcCWEI9BROS0lVfVkptj3Fraf+8dSRRakARjHncTu+JqG7DM3bea2QNmdmPQ7FFgpJnVAH8FxC8RvhuYCtzf5TLfAcBqM9sEbAB2At8N6xhERE5Xa3snK9bWc+15ozm7qDDqcvpEaJf/Arj7s8CzXZZ9LWG6Gbi1m+0eBB48yW4v7c0aRUR604vb9rDvSCu396N3sp+K7mwXEelFT1TWMm5YIdecmz0X+ShIRER6Sd3+Y7xWs4/b5kwgNwsG2eMUJCIivWRpVR0G3JYlg+xxChIRkV7Q3tHJsuo6PjZ9NOOGD4y6nD6lIBER6QUvvbmXvYdbKJuTXb0ROI0gMbOrzOwPguliM5scXlkiIpmlvKqO0UMHcO152fdA8qSCxMy+DtwD3Bcsygd+FFZRIiKZpOHgcV7ZvpfbSieQl5t9J3qSPeKbgBuBowDBk3mHhlWUiEgmWVZdhwOLsvC0FiQfJK3u7gQPVDSzweGVJCKSOTo6nWVVdVw1dRQTRgyKupxIJBsky8zsv4HhZvZF4Ofo0SQiIrz6ViMNTc1ZdSd7V0k9IsXd/9XMrgMOAdOBr7n7C6FWJiKSAZ6orGXUkALmnX921KVEJqkgCU5lveTuL5jZdGC6meW7e1u45YmIpK89h5p56c29fPHqKRTkZd8ge1yyR/4qMMDMxhM7rfUHwA/DKkpEJBP8pLqOjk7PyntHEiUbJObux4Cbgf9095uAC8IrS0QkvXV2OuVVdVz5kZFMGpXd1x8lHSRmdgXwWWIvnIKQH0EvIpLOXqvZR/2B45Rl8SB7XLJB8hfEXjq1Mng51WTgpfDKEhFJb+VVtZw1KJ/fm5G9g+xxyfYqjgGdwGIz+33ACO4pERHJNo2HW3h+6x6+cOUkBuTlRl1O5JINkh8Df0Ps/eid4ZUjIpL+Vqyrp73TKZub3YPscckGSaO7PxVqJSIiGcDdKa+sZe6kEUwdrSdFQfJB8nUz+x7wItASX+juK0OpSkQkTb2+433eff8Yfz5vWtSlpI1kg+QPgPOIPfU3fmrLAQWJiGSVJZV1FBXm8clZY6MuJW0kGyQXuvusUCsREUlz+4+2snrLbm6/bCKF+Rpkj0v28t9fm5luQBSRrLZyXT2tHZ0aZO8i2R7JVcAdZvYOsTESA9zdZ4dWmYhIGnF3llTWcvHE4Zw3pijqctJKskEyP9QqRETSXNW7B3i78SjfXKh/P3eV7GPk3wu7EBGRdFZeWcvQAXncMFuD7F2F+txjM5tvZtvNrMbM7u1m/QAzWxqsX2Nmk4Ll15nZWjPbHPy+NmGbS4PlNWb2LTOzMI9BRKTpWBvPbN7FgovHMahAjxnsKrQgMbNc4GHgE8SeFLy4mwH7O4ED7j4V+A/gG8HyfcCngyvF7gAeT9jmO8BdwLTgR6fdRCRUq9bX09LeSdkcPaCxO2H2SOYCNe6+w91bgXJgQZc2C4DHgunlwDwzM3df7+4NwfKtQGHQexkLFLn768E75P8H+EyIxyAiWc499rj42SXDmDl+WNTlpKUwg2Q8UJcwXx8s67aNu7cDTcDILm1uAda7e0vQvv4U+xQR6TXr6w7y5u7D6o30IMyTfd2NXXR9YnCPbcxsBrHTXdefxj7j295F7BQYEyfqL4CInJnyyloGFeRy40Xjoi4lbYXZI6kHEu/aKQEaTtbGzPKAYcD+YL4EWAV83t3fTmhfcop9AuDuj7h7qbuXFhcXp3goIpKNDje38dTGXdx44TiGDNAg+8mEGSRVwDQzm2xmBUAZUNGlTQWxwXSAhcBL7u5mNpzYmxjvc/dfxRu7+y7gsJldHlyt9XngyRCPQUSy2JMbGjje1sFivQWxR6EFSTDmcTewGtgGLAverviAmd0YNHsUGGlmNcBfEXsLI8F2U4H7zWxD8DM6WPcl4HtADfA28LOwjkFEstuSylrOH1vE7BINsvck1L6auz8LPNtl2dcSppuBW7vZ7kHgwZPssxqY2buVioicaHN9E1sbDvGPC2ag29V6FuoNiSIimeqJyloK83NYcLEuDD0VBYmISBdHW9qp2LCTG2aPo6gwP+py0p6CRESki6c2NnC0tYPFelx8UhQkIiJdLKmq49yzh3DJxLOiLiUjKEhERBK80XCIjXUHKZszUYPsSVKQiIgkKK+qpSAvh5sv0SB7shQkIiKB460drFq/k0/OHMPwQQVRl5MxFCQiIoFnNu/icHO77mQ/TQoSEZHAkspaphQPZu7kEVGXklEUJCIiwFt7DrP2vQOUzZmgQfbTpCARESHWG8nPNW65pOTUjeUEChIRyXrNbR2sXLeT62eMYeSQAVGXk3EUJCKS9Z7bspum423crkH2M6IgEZGst6SylokjBnHFlK5v+pZkKEhEJKu93XiENe/sp2zuBHJyNMh+JhQkIpLVllbVkZdjLLxUg+xnSkEiIlmrpb2D5Wvr+fj5ZzN6aGHU5WQsBYmIZK0X3tjD/qOtlOlx8SlRkIhI1iqvrGP88IFcPa046lIymoJERLLSe+8f5bWafSyaM4FcDbKnREEiIllpaVUdOQa3leq0VqoUJCKSddo6OllWXc+1541mzDANsqdKQSIiWefFbXvZd6RFj4vvJQoSEck6SyprGVNUyO+cq0H23qAgEZGsUn/gGK/+ppHb5kwgL1dfgb1Bf4oiklWWVdUBcFup7mTvLQoSEcka7cEg+++cW0zJWYOiLqffCDVIzGy+mW03sxozu7eb9QPMbGmwfo2ZTQqWjzSzl83siJl9u8s2rwT73BD8jA7zGESk/3hleyO7DzVTNkeD7L0pL6wdm1ku8DBwHVAPVJlZhbu/kdDsTuCAu081szLgG8AioBm4H5gZ/HT1WXevDqt2EemfyqtqKR46gHnn69+fvSnMHslcoMbdd7h7K1AOLOjSZgHwWDC9HJhnZubuR939NWKBIiKSsl1Nx3npzb3cemkJ+Rpk71Vh/mmOB+oS5uuDZd22cfd2oAlI5s0yPwhOa91vZt0+28DM7jKzajOrbmxsPP3qRaRf+Ul1PZ2OTmuFIMwg6e4L3s+gTVefdfdZwNXBz+e6a+Tuj7h7qbuXFhfrWnGRbNbR6SytquOqqaOYOFKD7L0tzCCpBxIfYlMCNJysjZnlAcOA/T3t1N13Br8PA08QO4UmItKt1vZO/vX57ew8eFx3sockzCCpAqaZ2WQzKwDKgIoubSqAO4LphcBL7n7SHomZ5ZnZqGA6H7gB2NLrlYtIv7BlZxM3fvs1vvPK29x08Xiun3F21CX1S6FdteXu7WZ2N7AayAW+7+5bzewBoNrdK4BHgcfNrIZYT6Qsvr2ZvQsUAQVm9hngeuA9YHUQIrnAz4HvhnUMIpKZWto7+M8Xa/jOL95mxOACvvv5Uq67QCESFuuhA9BvlJaWenW1rhYWyQYb6w7yt8s38taeI9x8yXi+dsMFDB9UEHVZGcnM1rp76anahdYjERHpS81tHfzfF3/Df//ibUYPLeT7Xyjl2vPUC+kLChIRyXjrag/wleWbqNl7hNtKS/jqpy5g2MD8qMvKGgoSEclYzW0d/PsLb/G9X+5gTFEhj/3hXD0aPgIKEhHJSNXv7ucryzexY99RFs+dyN998jyGFqoXEgUFiYhklOOtHTy0ejs/+N93GDdsID+68zKumjYq6rKymoJERDLGmh3v85UVm3jv/WN87vJzuOcT5zFkgL7Goqb/AiKS9o62tPPN597ksdffY8KIgTzxxcu48iPqhaQLBYmIpLX/rdnHPSs3Ubf/OF+4chJfmT+dQQX66kon+q8hImnpSEs7//Kzbfzo17VMGjmIZX98BXMnj4i6LOmGgkRE0s5rv9nHPSs20dB0nD+6ajJ/ff10BhbkRl2WnISCRETSxqHmNv752W0sqaxjSvFglv/JFVx6jnoh6U5BIiJp4ZXte7lv5Wb2HGrmj6+ZwpevO5fCfPVCMoGCREQi1XS8jQeffoOfrK1n6ughrPjSlVw88ayoy5LToCARkci8uG0Pf7dqM/uOtPKnH/sIfz5vmnohGUhBIiJ97uCxVh546g1Wrt/J9LOH8t3PlzK7ZHjUZckZUpCISJ9avXU3/+enWzhwtJU/v3Yqf3btVAbkqReSyRQkItIn9h9t5e8rtlKxsYHzxxbxgy/MYeb4YVGXJb1AQSIiofvZ5l3c/+QWDh5r48sfP5cvfewjFOTlRF2W9BIFiYiEZt+RFr7+5Fae2byLmeOLePzOyzh/bFHUZUkvU5CISK9zd57etIuvV2zlcHMbf/t707nrmink56oX0h8pSESkVzUebuH+n27hua27mV0yjIcWXs70MUOjLktCpCARkV7h7jy5oYG/f2orx1o7uGf+eXzx6snkqRfS7ylIRCRlew8183ertvDzbXu4eOJwHlo4m6mj1QvJFgoSETlj7s6KdTt54KmttLR38tVPns8fXjWZ3ByLujTpQwoSETkju5uauW/lJl7e3kjpOWfxzYWzmVI8JOqyJAIKEhE5Le7OT6rr+cdn3qCto5Ov3XABd1w5Sb2QLBbqKJiZzTez7WZWY2b3drN+gJktDdavMbNJwfKRZvaymR0xs2932eZSM9scbPMtM9PfXpE+svPgce74QRVfWbGJ88cW8dxfXKNTWRJej8TMcoGHgeuAeqDKzCrc/Y2EZncCB9x9qpmVAd8AFgHNwP3AzOAn0XeAu4BfA88C84GfhXUcIhLrhSyprOOfnt1GpzsPLJjB7192DjkKECHcU1tzgRp33wFgZuXAAiAxSBYAfx9MLwe+bWbm7keB18xsauIOzWwsUOTurwfz/wN8BgWJSGjq9h/j3pWb+FXN+1wxZSTfXDibCSMGRV2WpJEwg2Q8UJcwXw9cdrI27t5uZk3ASGBfD/us77LP8b1SrYicoLPT+fGa9/jnn72JAQ9+Zia3z52oXoh8SJhB0t3fNj+DNmfU3szuInYKjIkTJ/awSxHp6r33j3LPik38esd+rp42in++eRYlZ6kXIt0LM0jqgQkJ8yVAw0na1JtZHjAM2H+KfZacYp8AuPsjwCMApaWlPYWTiAQ6O53HXn+Xbz63nbwc419unsWiORPQNS3SkzCDpAqYZmaTgZ1AGXB7lzYVwB3A68BC4CV3P+mXvrvvMrPDZnY5sAb4PPCfYRQvkm3e2XeUe5ZvovLd/XxsejH/dNMsxg0fGHVZkgFCC5JgzONuYDWQC3zf3bea2QNAtbtXAI8Cj5tZDbGeSFl8ezN7FygCCszsM8D1wRVfXwJ+CAwkNsiugXaRFHR0Oj/41Tv86/Pbyc/N4aGFs1l4aYl6IZI066ED0G+UlpZ6dXV11GWIpJ23G4/wtz/ZyLrag8w7bzT/dPMszi4qjLosSRNmttbdS0/VTne2i2Shjk7ne7/cwb+98BYD83P5j0UX8pmLxqsXImdEQSLST7g7zW2dHGpu49DxtuB3+2/nm463cai5nUPH29hU38Qbuw5x/QVn8+BNMxk9VL0QOXMKEpE00tzW8aEAiH/5N3UTDoea2zmcsK6to+dT1YX5ORQV5jNicAHfWnwxn549Vr0QSZmCRKQXtbZ3figA4l/+HwRB13Wx+abjbbS2d/a4/4LcHIoG5lM0MI+iwnyGDcxnwlkDGTYwP7a88IN1sfm8364bWpjHgLzcPvqTkGyiIOnBvz2/nb2HWsjJATMjxyDHjBwz7LfTBPPxZR/M5yS0sWC7D9YnbJ/TTXsS2uSc7j4T13/494eOIefEfRoJ2wR1WMK6+LbxdvHpD5YblsPJ95NwbLE2pM2/its6OjncJQBOPDXUfW/hUHNsfXNbz0GQl2MJX/p5FA3MZ9zwgd0GQDwYhiUER2G+gkDSj4KkB2vfO8COxqN0utPpsXPQ8elOdzz4feL6D37L6ek2kBLC58Sg+yDEugZSfDonIRR72g/AsdYOmo63cay1o8cac3PshC/5ooF5nF00pNsASOw1xJcV5uekTWiK9BYFSQ+e+OLlKW3vCaFzYvAEQdTZfRAltj9ZWJ10n/Hpzi7hRpc2nfT8ucT3HVvvwfHE1pHQLmH5bz8rsW3CfvigXY/7IV7Xh7f50H744M8oPn3K/XDiceEweEDuScLgxB7BoIJcBYFIFwqSEJkZuQa53T4iTESkfwj1xVYiItL/KUhERCQlChIREUmJgkRERFKiIBERkZQoSEREJCUKEhERSYmCREREUpIVL7Yys0bgvTPcfBSwrxfLyQQ65uyQbcecbccLqR/zOe5efKpGWREkqTCz6mTeENaf6JizQ7Ydc7YdL/TdMevUloiIpERBIiIiKVGQnNojURcQAR1zdsi2Y86244U+OmaNkYiISErUIxERkZQoSJJgZg+Z2ZtmtsnMVpnZ8KhrCpuZ3WpmW82s08z67ZUuZjbfzLabWY2Z3Rt1PWEzs++b2V4z2xJ1LX3FzCaY2ctmti34O/0XUdcUNjMrNLNKM9sYHPM/hPl5CpLkvADMdPfZwFvAfRHX0xe2ADcDr0ZdSFjMLBd4GPgEcAGw2MwuiLaq0P0QmB91EX2sHfhrdz8fuBz4syz479wCXOvuFwIXAfPNLLVXvvZAQZIEd3/e3duD2V8DJVHW0xfcfZu7b4+6jpDNBWrcfYe7twLlwIKIawqVu78K7I+6jr7k7rvcfV0wfRjYBoyPtqpwecyRYDY/+AltQFxBcvr+EPhZ1EVIrxgP1CXM19PPv2CynZlNAi4G1kRbSfjMLNfMNgB7gRfcPbRj1jvbA2b2c2BMN6u+6u5PBm2+Sqyb/OO+rC0syRxzP2fdLNNljP2UmQ0BVgB/6e6Hoq4nbO7eAVwUjOmuMrOZ7h7K2JiCJODuH+9pvZndAdwAzPN+cs30qY45C9QDExLmS4CGiGqREJlZPrEQ+bG7r4y6nr7k7gfN7BViY2OhBIk8bkiuAAADY0lEQVRObSXBzOYD9wA3uvuxqOuRXlMFTDOzyWZWAJQBFRHXJL3MzAx4FNjm7v8edT19wcyK41eXmtlA4OPAm2F9noIkOd8GhgIvmNkGM/t/URcUNjO7yczqgSuAZ8xsddQ19bbgAoq7gdXEBmCXufvWaKsKl5ktAV4HpptZvZndGXVNfeCjwOeAa4P/fzeY2SejLipkY4GXzWwTsX8wveDuT4f1YbqzXUREUqIeiYiIpERBIiIiKVGQiIhIShQkIiKSEgWJiIikREEi0gMzO3LqVj1uv9zMppyizSunesJyMm26tC82s+eSbS+SCgWJSEjMbAaQ6+47+vqz3b0R2GVmH+3rz5bsoyARSYLFPGRmW8xss5ktCpbnmNl/Be98eNrMnjWzhcFmnwWeTNjHd8ysuqf3Q5jZETP7NzNbZ2Yvmllxwupbg3dMvGVmVwftJ5nZL4P268zsyoT2Pw1qEAmVgkQkOTcTe6/DhcQeN/GQmY0Nlk8CZgF/ROxJAHEfBdYmzH/V3UuB2cDvmNnsbj5nMLDO3S8BfgF8PWFdnrvPBf4yYfle4Lqg/SLgWwntq4GrT/9QRU6PHtookpyrgCXBE1X3mNkvgDnB8p+4eyew28xeTthmLNCYMH+bmd1F7P+7scReprWpy+d0AkuD6R8BiQ8YjE+vJRZeEHvPxLfN7CKgAzg3of1eYNxpHqfIaVOQiCSnu0fO97Qc4DhQCGBmk4G/Aea4+wEz+2F83SkkPsOoJfjdwQf/734Z2EOsp5QDNCe0LwxqEAmVTm2JJOdVYFHwsqBi4BqgEngNuCUYKzkb+FjCNtuAqcF0EXAUaArafeIkn5MDxMdYbg/235NhwK6gR/Q5IDdh3bmE9NhwkUTqkYgkZxWx8Y+NxHoJX3H33Wa2AphH7Av7LWJv3msKtnmGWLD83N03mtl6YCuwA/jVST7nKDDDzNYG+1l0irr+C1hhZrcCLwfbx/1uUINIqPT0X5EUmdkQdz9iZiOJ9VI+GoTMQGJf7h8NxlaS2dcRdx/SS3W9Cixw9wO9sT+Rk1GPRCR1TwcvESoA/tHddwO4+3Ez+zqx98DX9mVBwem3f1eISF9Qj0RERFKiwXYREUmJgkRERFKiIBERkZQoSEREJCUKEhERSYmCREREUvL/Ab3rJ+8gDiW2AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Plot important coefficients\n",
    "coefs = pd.Series(ridge.coef_, index = feat_names)\n",
    "print(\"Ridge picked \" + str(sum(coefs != 0)) + \" features and eliminated the other \" +  \\\n",
    "      str(sum(coefs == 0)) + \" features\")\n",
    "\n",
    "#正系数值最大的10个特征和负系数值最小（绝对值大）的10个特征\n",
    "imp_coefs = pd.concat([coefs.sort_values().head(10),\n",
    "                     coefs.sort_values().tail(10)])\n",
    "imp_coefs.plot(kind = \"barh\")\n",
    "plt.title(\"Coefficients in the Ridge Model\")\n",
    "plt.show()\n",
    "\n",
    "mse_mean = np.mean(ridge.cv_values_, axis = 0)\n",
    "plt.plot(np.log10(alphas), mse_mean.reshape(len(alphas),1)) \n",
    "\n",
    "plt.xlabel('log(alpha)')\n",
    "plt.ylabel('mse')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "instant         0.044471\n",
       "season_1       -0.093325\n",
       "season_2        0.007500\n",
       "season_3        0.010736\n",
       "season_4        0.075089\n",
       "mnth_1         -0.035170\n",
       "mnth_2         -0.022045\n",
       "mnth_3          0.027077\n",
       "mnth_4          0.003468\n",
       "mnth_5          0.052737\n",
       "mnth_6          0.012862\n",
       "mnth_7         -0.035654\n",
       "mnth_8          0.012665\n",
       "mnth_9          0.073244\n",
       "mnth_10         0.021396\n",
       "mnth_11        -0.056884\n",
       "mnth_12        -0.053695\n",
       "weathersit_1    0.087061\n",
       "weathersit_2    0.039426\n",
       "weathersit_3   -0.126487\n",
       "weekday_0      -0.019192\n",
       "weekday_1      -0.017963\n",
       "weekday_2      -0.005523\n",
       "weekday_3      -0.000532\n",
       "weekday_4       0.007373\n",
       "weekday_5       0.010071\n",
       "weekday_6       0.025766\n",
       "temp            0.217209\n",
       "atemp           0.184670\n",
       "hum            -0.175380\n",
       "windspeed      -0.153843\n",
       "holiday        -0.023324\n",
       "workingday      0.016749\n",
       "yr              0.202449\n",
       "dtype: float64"
      ]
     },
     "execution_count": 51,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "coefs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "相比OLS，岭回归模型增加了L2正则，系数值进行了收缩。\n",
    "由于增加正则限制了模型复杂的，相比比OLS模型，岭回归模型在训练集上的误差略有增大，但在测试集上的误差有所减小。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_cell_guid": "2da33461-fb1a-c09a-b0fe-d20e8c8f52e3",
    "_uuid": "d2e8bab153344d6ecf2e48909b76bd8f387e2710"
   },
   "source": [
    "**3* Linear Regression with Lasso regularization (L1 penalty)**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {
    "_cell_guid": "8525724a-fb77-66d4-e06f-f3365fbd8ec3",
    "_uuid": "f8f41944e405693ba867158353ffedcd9ecd4a07",
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Best alpha : 0.00026880501220132827\n",
      "cv of rmse : 0.09532386631968247\n",
      "Lasso picked 27 features and eliminated the other 7 features\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAacAAAEICAYAAAD7pTujAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzt3Xu4VVW9//H3J+8KSSqZ5oUUjVQMY6t5lAIl7WJaJ3y8VWKePGbm6WepHPUYZZZGaWaaqZk3jpmmpWneMAQRUFBgQ94FD6QpkngpJZXv748xlky2+7L23muvy96f1/Psx7XmHHPOMdbG9d1jzDG/QxGBmZlZPXlXrStgZmbWkoOTmZnVHQcnMzOrOw5OZmZWdxyczMys7jg4mZlZ3XFwsoYm6YOSHpL0iqTjJa0n6WZJL0m6TtLhku4o4zynSLq0GnVupw5bSXpV0hoVOt94SVdX4lx9jaRFkkaXUW6QpJC0ZjXq1Zc4OFlVSDpM0qz85fuspD9J2qsCpz4JmBwR/SPiZ8AYYFNg44g4KCImRsS+HZ0kIn4QEf/R3cp058sqIv4vIvpFxFtduO5ISUs6e1wnzn+5pO/31Pm7KtcrJB3QYvtP8/axNaqadZODk/U4SScAPwV+QAocWwEXAgdW4PRbAwtavH8sIt6swLmtMTwGHFF6k/8wOAh4smY1sm5zcLIeJWlD4HvA1yPihoj4R0S8ERE3R8SJucw6+S/dZ/LPTyWtUzjH/pLmSFou6T5JO+ftdwOjgJ/nHtk1wOnAwfn9UZLGSrq3cK4dJd0p6e+SnpN0St6+2hCYpI/may2XNFfSyMK+yZLOkDQtDyfeIWmTvHtK/u/yXIc9JA2WdE8eanxB0rVtfFar9bo6uE7xuA2APwGb52u+KmnzvHttSVfm4xdIaioct7mk30laKmmhpOPL+Z22cv3zJC2W9LKk2ZJGFPbtlnvML+fP+5y8fV1JV0talj/jByRtWqjXTfl39ISkr3ZQhZuBPSW9J7//JDAP+FuhHu+SdJqkpyU9nz+TDQv7v5T3LZN0aov2vUvSOElP5v2/lbRRVz4rK5+Dk/W0PYB1gRvbKXMq8FFgGPBhYDfgNABJHwEuA/4T2Bj4JXCTpHUiYm9gKnBcHg47lNQ7uza//1XxIpL6A3cBtwGbA4OBSS0rI+n9wC3A94GNgG8Dv5M0sFDsMOBI4L3A2rkMwMfyfwfkOkwHzgDuAN4DbAGc385n0VJb13lbRPwD+BTwTL5mv4h4Ju8+APgNMAC4Cfh5buO7SF/qc4H3A/sA35S0XyfqVvIA6Xe3EfC/wHWS1s37zgPOi4h3A9sCv83bjwA2BLYk/V6PAV7L+64BlpB+R2OAH0jap53rv57bdkh+/2XgyhZlxuafUcA2QD9WfRY7AL8AvpSvuTHp91RyPPA54ON5/4vABe3UxyrAwcl62sbACx0Msx0OfC8ino+IpcB3SV8UAF8FfhkRMyPirYi4AlhBCmadtT/wt4j4SUS8HhGvRMTMVsp9Ebg1Im6NiJURcScwC/h0ocyvI+KxiHiN9IU7rJ3rvkEabtw8X/fedsq21JnrtObe3I63gKtIwR9gV2BgRHwvIv4VEU8Bl7DqC75sEXF1RCyLiDcj4ifAOsAH8+43gMGSNomIVyNiRmH7xsDg/HudHREvS9oS2As4OX9Wc4BLWfXvoS1XAl/OvaGPA79vsf9w4JyIeCoiXgX+Gzgk91LHAH+MiCkRsQL4H2Bl4dj/BE6NiCV5/3hgjDwJokc5OFlPWwZs0sH/yJsDTxfeP523QfpS/1Ye+lkuaTnpr+3N6bwtKe8+xNbAQS2uuRewWaHM3wqv/0n6S7wtJwEC7s9Da1/pRJ07c51yjl83/y62Jg0DFtt4CumeYKdI+pakh/Ow5XJSj6g0/HgUsD3wSB662z9vvwq4HfiN0lDujyStRfq9/j0iXilc4mlS765NOeAPJPW4/5iDeVFr/8bWzO3dHFhcONc/SP9uS7YGbix8Tg8Db9GFz8rK58hvPW06adjlc8D1bZR5htUnNmyVt0H60jgzIs6sQF0WA4eWWe6qiOjoXkdr3pHmPyL+RuoBojRD8S5JUyLiiS6cv+zrdmAxsDAituvORfP9pZNJw4ILImKlpBdJwZiIeBw4NA8j/jtwvaSNcwD4LvBdSYOAW4FHScOfG0nqXwhQWwF/LaM6V5PuOY5qZV/p31jJVsCbwHPAs8CHCm1an9SrK1kMfCUiprXS/kFl1Mu6wD0n61ER8RLpC+MCSZ+TtL6ktSR9StKPcrFrgNMkDcw3/E8nfdFAGmo6RtLuSjaQ9Jl8/6iz/gi8T9I3lSZh9Je0eyvlrgY+K2k/SWvkm/cjJW3RStmWlpKGhLYpbZB0UOHYF0mBpNPTxTvwHLBx8SZ/B+4HXpZ0stKzYWtI2knSru0cU/osSj9rA/1JX/JLgTUlnQ68u3SApC9KGhgRK4HlefNbkkZJGqr0TNfLpGG+tyJiMXAf8MN8jZ1Jva+JZbTpZ8AnWDUppega4P9J+oCkfqy6N/km6Y+m/SXtldv0PVb/brwIOFPS1rlNAyVVYqaptcPByXpcRJwDnEAacllK+kv0OFbdF/g+6Z7OPKAZeDBvIyJmkXodPyd9sT9BurHdlXq8Qvry+ixpuOtxWvkrO39BHkga5irV90TK+P8lIv4JnAlMy8NAHyXd35kp6VXSjfv/ioiFXWlDO9d9hPQF/FS+brvDnvke1GdJ97AWAi+Q7u20F9zGkSYtlH7uJg3N/Yk0nftpUi95ceGYTwILctvPAw6JiNeB95GCwsukYbJ7WPUHyaHAIFJv50bgO/m+X0efwd8jYlK0vkjdZaShxCm5va8D38jHLQC+TprM8Szp31nxmbHzSL+3OyS9AswAWvujxipIXmzQzMzqjXtOZmZWdxyczMys7jg4mZlZ3XFwMjOzuuPnnLpok002iUGDBtW6GmZmDWX27NkvRMTAjso5OHXRoEGDmDVrVq2rYWbWUCQ93XEpD+uZmVkdatiek6QBwGERcWGt62K926S7t611Fczqyj579/xSWY3ccxoAHFvrSpiZWeU1cnA6C9hWaRG6CZJOzFmP50n6Lry9eNsjki6VNF/SREmjlRZve1zSbrnceElXSbo7b+9Kwk8zM6uQRg5O44AnI2IYcCewHWmRumHAcEmlRd8Gk3Jj7QwMIS3ethdp0bZTCufbGfgMaXG801vLTSbpaKVVPWctXbq0Z1plZmYNHZyK9s0/D5GShg4hBStIywI056zIC4BSYshmUnLJkj9ExGsR8QLwZ1KgW01EXBwRTRHRNHBghzMhzcysixp2QkQLAn4YEb9cbWNaa2VFYdPKwvuVrN7+lhlwnRHXzKxGGjk4vUJaSwZS2v4zJE2MiFclvZ+0PkxnHCjph8AGwEjSsKFZVWYmmdnqGjY4RcSyPLFhPmk9mf8FpksCeBX4Ip1b0O1+4BbSCplnRMQzHZQ3M7Me0rDBCSAiDmux6bxWiu1UKD+28HpRcR/wWEQcXcn6mZlZ1/SWCRFmZtaLNHTPqVIiYnyt62BmZqs4OJl1YPz48bWugtUB/zuoLg/rtUHSGrWug5lZX9Vne06SzgBeiIjz8vszgeeAzwPPkjJN7FC7GpqZ9V19uef0K+AIAEnvAg4B/krKDHFqRLwjMDl9kZlZdfTZ4JSnki+TtAurUh8tA+6PiIVtHOP0RWZmVdBnh/WyS4GxwPuAy/K2f9SsNlaXfCPcrPr6bM8puxH4JLArKQWSmZnVgT7dc4qIf0n6M7A8It7KqY/MzKzG+nRwyhMhPgocBBARk4HJNaySmZnRh4f1JO0APEFa3+nxWtfHzMxWadjgJOmUjku1LSL+EhHbRMS3KlUnMzOrjEYe1jsF+EGtK2G935JxU2tdharY4qwRta6C2dsaouck6feSZktakB+EPQtYT9IcSRNzmS9Kuj9v+2Up/ZCkVyWdnY+/S9JukiZLekrSAbnMWEl/kHSbpEclfaeGzTUz6/MaIjgBX4mI4UATcDwwAXgtIoZFxOGSPgQcDOwZEcNIiwweno/dAJicj38F+D7wCVKaou8VrrFbPmYYcJCkpiq0y8zMWtEow3rHS/p8fr0lsF2L/fsAw4EH8nTw9YDn875/Abfl183Aioh4Q1IzMKhwjjsjYhmApBuAvYBZxYtIOho4GmCrrbbqfqvMzKxVdR+cJI0ERgN7RMQ/JU0G1m1ZDLgiIv67lVO8ERGRX68EVgBExEpJxfZHi+NaviciLgYuBmhqanrHfjMzq4y6D07AhsCLOTANIT2XBPCGpLUi4g1gEvAHSedGxPOSNgL6R8TTnbjOJ/JxrwGfA75SyUZY4/JEAbPqa4R7TrcBa0qaB5wBzMjbLwbmSZoYEX8BTgPuyOXuBDbr5HXuBa4C5gC/i4hZHZQ3M7MeolUjXn2XpLFAU0QcV+4xTU1NMWuW45eZWWdImh0RHU44a4Sek5mZ9TGNcM+px0XE5cDlNa6GmZll7jmZmVnd6TU9J0kDgMMi4sL8fiTw7YjYv8zj9wZ+DKwNzAaOiog3e6i61kB+cnBZ/4Tqwreu/WOtq2BWEb2p5zQAOLYrB+alM64ADomInYCngSMqWDczM+uEugpOkgZJekTSpZLmS5ooabSkaZIez3nxxku6rJAf7/h8+FnAtjm33oS8rZ+k6/M5J6rt1QQ3JmWOeCy/vxP4Qiv1O1rSLEmzli5dWtG2m5nZKnUVnLLBwHnAzsAQ4DBSKqFvkzKRk7fvR8qH9x1JawHjgCdzvr0Tc7ldgG8COwDbAHu2cc0XgLUK+fTGkNIkrSYiLo6IpohoGjhwYPdaaWZmbarH4LQwIpojYiWwgLQYYJDy4g3KZW6JiBUR8QIph96mbZzr/ohYks81h9Vz6b0tn/8Q4FxJ95MSxPp+k5lZjdTjhIgVhdcrC+9Xsqq+xTJv0XY7yi1HREwHRgBI2hfYvvwqW2/mSQZm1VePPaeuegXo39WDJb03/3cd4GTgogrVy8zMOqnXBKe83MW0PJFiQocHvNOJkh4G5gE3R8Tdla2hmZmVy7n1usi59czMOs+59czMrGHV44SIHiXpRuADLTafHBG316I+Zmb2Tn0uOEXE59vbL+kmYJucKcKMC46pze3Hr1+0d02ua1YPPKxXIOnfgVdrXQ8zs76uasFJ0gaSbpE0N8+oO1jScEn3SJot6XZJm+WyX5X0QC77O0nr5+0H5WPnSpqSt60r6deSmiU9JGlU3j5W0g2Sbsupj37UQf36AScA32+njNMXmZlVQTV7Tp8EnomID+chs9uA84ExETEcuAw4M5e9ISJ2jYgPAw8DR+XtpwP75e0H5G1fB4iIocChwBWS1s37hgEHA0OBgyW9IyVRwRnAT4B/tlXA6YvMzKqjmsGpGRgt6WxJI0i563YC7pQ0BzgN2CKX3UnSVEnNwOHAjnn7NOBySV8F1sjb9gKuAoiIR0gZxUvZHSZFxEsR8TrwF2Dr1iomaRgwOCJurFxzzcysq6o2ISIiHpM0HPg08ENS5u8FEbFHK8UvBz4XEXMljQVG5nMcI2l34DPAnBxU2so0DuWnL9oDGC5pUS7zXkmTI2Jkea2z3swTE8yqr5r3nDYH/hkRV5MW9dsdGChpj7x/LUmlHlJ/4Nmcbfzwwjm2jYiZEXE6KZP4lsCUUhlJ2wNbAY92pm4R8YuI2DwiBpF6Yo85MJmZ1U41p5IPBSZIWgm8AXyNlPn7Z5I2zHX5KSkT+f8AM0lDdM2sypk3QdJ2pN7SJGAu8AhwUR4CfBMYGxEr2l66yczM6p3TF3WR0xeZmXWe0xeZmVnD6nMZIiTNBNZpsflLEdFci/pY/Xt4yIeqcp0PPfJwVa5j1giq3nPKD8duXni/SNImPXCdWyUNyD/HlrZHxO55KffiT3N+WHe5JK8sZ2ZWY7UY1hsLbN5RoXJIam9l209HxHJgAHBsW+UKJgBfqkS9zMysezoMTpJOknR8fn2upLvz630kXS1pX0nTJT0o6bqcBghJp+cURPMlXaxkDNAETJQ0R9J6+TLfyMc3SxqSj99A0mX5HA9JOjBvH5uvczNwh6TNJE3J55ufH/At9sjOArbN+9tchDAiJpFW023vs3D6IjOzKiin5zQFGJFfNwH98vNHe5GmeZ8GjI6IjwCzSPnpAH6eUxDtBKwH7B8R1+cyh+fhtNdy2Rfy8b8Avp23nQrcHRG7AqNI08g3yPv2AI6IiL2Bw4DbI2IY8GFgTov6jwOezNc7sZwPpS1OX2RmVh3lTIiYTcqe0J+UceFBUpAaAdwE7EBaHh1gbWB6Pm6UpJOA9YGNSM8v3dzGNW4oXOvf8+t9gQMklYLVuqQHbAHujIi/59cPAJflgPn7iGgZnMzMrMF0GJwi4o2c1udI4D5gHqknsy2wkBQoDi0ekxOvXgg0RcRiSeNJwaUtpTRDxRRDAr4QEatle8jpi/5RqN8USR8jpTS6StKEiLiyo3aZlcuz6Myqr9wJEVNIw21TgKnAMaThsxnAnpIGA0haP6cQKgWiF/I9qDGFc73CqowP7bmddC9K+dy7tFZI0tbA8xFxCfAr4CMtipR7PTMzqxPlBqepwGbA9Ih4DngdmBoRS0mz766RNI8UrIbkWXKXkO5J/Z409FZyOSndUHFCRGvOANYC5kman9+3ZiQpCexDwBeA84o7I2IZadhxfnsTIiRNBa4D9pG0RNJ+7dTNzMx6kNMXdZHTF5mZdZ7TF5mZWcPqU+mLJA0lL0xYsCIidq9FfawxDL1iaI9fo/kIZ88yK+o1PaeWaYokjWyZiigimltJXbR7Ln+5pIX5XlhpIUMzM6uBXhOcKD9NUXtOLAQtPy9lZlYjdRWcJA2S9IikS/PsuomSRkuaJulxSbtJGp/TGk2W9FQptRKtpynqJ+n6fM6JpWnp3aif0xeZmVVBXQWnbDBpOvjOwBBSeqK9SM9ZnZLLDAH2A3YDvpOzQ7SWpmgX4JukLBbbAHt2cO0zJc3LOQRbLqvh9EVmZlVSj8FpYb43tJKU8mhSpPnuzcCgXOaWiFgRES8AzwObtnGu+yNiST7XnMLxrflvUtDblZRu6eRut8TMzLqkHmfrrSi8Xll4v5JV9S2WKaY8au9c7ZUjIp4tHSPp16xKQGt9nGfSmVVfPfacuqpbaYokbZb/K+BzwPwK1cvMzDqpHntOXRIRy/LEifnAn4BbOnmKiZIGkhLOziHlDzQzsxpw+qIucvoiM7POc/oiMzNrWL1mWK9ckm4EPtBi88kRcXst6mMNYPyGPXz+l3r2/GYNqKF6TuWkKOrg+OOAoaTl3EeXskEAd0j6maQn8nNOLdeEMjOzKmqo4ET3UxRNA0YDT7fY/ilgu/xzNPCLblzDzMy6qerBqZYpiiLioYhY1MquA4ErI5kBDChNLW9Rd6cvMjOrglr1nGqZoqg17wcWF94vydtW4/RFZmbVUavgVKsURW1prbflOfZmZjVSq9l6NUlR1I4lwJaF91sAz3ThPNYbeTadWdU12oSIbqUoasdNwJeVfBR4qZBrz8zMqqyhglNELAOm5YkUEzo8oAVJx0taQuoZzZN0ad51K/AU8ARwCd1ftNDMzLrB6Yu6yOmLzMw6z+mLzMysYfXK9EVOUWSVNGhcZxPct27RWZ+pyHnM+oJe03MqpjaKiM+Tnn1aUkpR1FFgkvQrSXNz+qLrJfWrRr3NzOydek1wovupjf5fRHw4InYG/g84rjLVMjOzzqqr4FTj1EYv5zoIWI9WHsJ1+iIzs+qoq+CU1Sy1kaRfA3/L5z+/5X6nLzIzq456nBCxMCKaASS9ndpIUim10RxyaiNghaQOUxvlc5VSG93b1oUj4khJa5AC08HAryvTJGtknshgVn312HOqaWqjiHgLuBb4QjmVNTOzyqvH4NRVXU5tlNMWDS69Bj4LPFLBupmZWSfU47Bel0TEsjxxYj7wJ6AzD6cIuELSu/PrucDXeqCaZmZWBqcv6iKnLzIz6zynLzIzs4ZVd8N6kiYD346INrslksYCTRHR6Qdl20ptBGwIjCc93zQ3Ig7r7Lmtd2orfZFn8Zn1nLoLTj0tpzZajaTtgN8Ce0bEi5LeW/2amZlZSbeH9SSdVMrSIOlcSXfn1/tIulrSvpKmS3pQ0nWlnHWShku6R9JsSbdL2qzFed8l6QpJ38/vj5T0mKR7KDxMK+mzkmZKekjSXZI2zcc+Lmlg4VxPSNqkjWZ8FbggIl4EiIjnu/u5mJlZ11XintMUYER+3URKGbQWKatDM3AaMDoiPgLMAk7I+88HxkTEcOAy4MzCOdcEJgKPRcRpOXB9lxSUPkHK+FByL/DRiNgF+A1wUkSsBK4GDs9lRpOG6l5oow3bA9vn2X4zJH2ytUJOX2RmVh2VGNabDQyX1J/00OuDpCA1grT8+Q6k1WsB1gamAx8EdgLuzNvXAIrLov8S+G1ElALW7sDkiFgKIOlaUkCBtKrttTmArQ0szNsvA/4A/BT4Cu1ne1gT2A4Ymc83VdJOEbG8WCgiLgYuhjRbr+OPxszMuqLbwSki3pC0CDgSuA+YB4wCtiUFijsj4tDiMZKGAgsiYo82TnsfMErSTyLi9dKl2ih7PnBORNwkaSRpUgMRsVjSc5L2JgW3w9s4HmAJMCMi3gAWSnqUFKweaOcY6yM88cGs+io1lXwKKTHrFGAqcAwpB94MYM9C9oX1JW0PPAoMlLRH3r6WpB0L5/sVcCtwnaQ1gZnASEkb5yHBgwplNwT+ml8f0aJel5KG936b0xK15fekgEq+L7U98FQn2m9mZhVUqeA0FdgMmB4RzwGvA1PzMNxY4BpJ80jBakhE/AsYA5wtaS4pkP1b8YQRcQ5piPAq4DlSj2g6cFfeXjKeFMSmAi3vKd0E9KPjBK63A8sk/QX4M3BiRCwrt/FmZlZZvTpDhKQm4NyIGNFh4U5yhggzs84rN0NEr33OSdI4Un689u41mZlZHeq1wSkiziKtjvs2Saey+v0qgOsKswLNzKwO9JrgJGkAcFhEXJjfjySlQdq/VCYHoVYDkaTjSKvmbgsMbOeZKOsl3vfnOWWV+9uoYT1cEzNrqTclfh0AHNuN46eRHtZ9ujLVMTOzrqqr4CRpkKRHJF0qab6kiZJG58wNj0vaTdJ4SZdJmizpqVLqJNIQ3raS5kiakLf1k3R9PufEvJBgqyLioYhY1NNtNDOzjtXjsN5g0n2ho0kPwR5GSoV0AHAKadr5ENJzSf2BRyX9AhgH7BQRw+DtYb1dgB2BZ0g9oz1J6Y66RNLRuV5stdVWXT2NmZl1oK56TtnCiGjO+fEWAJMizXdvBgblMrdExIp8X+h5YNM2znV/RCzJ55pTOL5LIuLiiGiKiKaBAwd251RmZtaOeuw5rSi8Xll4v5JV9S2WeYu221FuOeuDPNHBrH7VY8+pq14hDfOZmVmD6zXBKacbmpYnUkzo8IAWJB0vaQkpK/k8SZdWvJJmZlaWXp2+qCc5fZGZWeeVm76o1/SczMys9+hzEwQk3Qh8oMXmkyPi9lrUx8zM3qmhglM5KYo6OP44YCgtUhRJOjwv6QHwKvC1iJhb6fpbfZl097Zlldtn7yd7uCZm1lKjDev1VIqihcDHI2Jn4AzyUuxmZlYbVQ9O9ZiiKCLui4gX89sZpBl7rdX9aEmzJM1aunRp1z8EMzNrV616ToOB84CdSamISimKvk1KUUTevh+wG/CdvDz7OODJiBgWESfmcruQsonvAGxDSlHUHUcBf2pthzNEmJlVR62CU12mKJI0ihScTu7qOczMrPtqNSGi7lIUSdoZuBT4VH6g13o5T3Qwq1+NNiGiR1IUSdoKuAH4UkQ8Vunzm5lZ5zRUcOrBFEWnAxsDF+bJFk79YGZWQ05f1EVOX2Rm1nlOX2RmZg2roTJElMspiszMGltDBady0xdFxOfbOP440jNRLdMXDQF+DXwEODUiftxjjbCaGT9+fFWPM7Oua7RhvZ5KX/R34HjAQcnMrA44fVHa/nxEPAC80UHdnb7IzKwKnL6oE5y+yMysOpy+yMzM6o7TF1mf4YkNZo2j0SZE9Ej6IjMzqy8NFZx6Kn2RpPfl7ScAp0laIundFa28mZmVzemLusjpi8zMOs/pi8zMrGH1yskDTl9kAEvGTa3IebY4a0RFzmNm5au74CRpMiklUZtjZpLGAk0RcVxr+9tKX9TO+dYBrgSGA8uAg1t7WNfMzKrDw3rJUcCLETEYOBc4u8b1MTPr07odnCSdVEovJOlcSXfn1/tIulrSvpKmS3pQ0nWS+uX9wyXdI2m2pNslbdbivO+SdIWk7+f3R0p6TNI9FLJASPqspJmSHpJ0l6RN87GPSxpYONcTkjZpoxkHAlfk19cD+7SWBsnpi8zMqqMSPacpQGlQvomU624tUjqiZuA0YHREfASYBZyQ958PjImI4cBlwJmFc64JTAQei4jTcuD6LikofYKUqqjkXuCjEbEL8BvgpJwt4mrg8FxmNDC3lIW8Fe8HFgNExJvAS6SVcVfj9EVmZtVRiXtOs4HhkvqTsjU8SApSI4CbSIFkWu6IrA1MBz4I7ATcmbevATxbOOcvgd9GRClg7Q5MjoilAJKuBbbP+7YArs0BbG1gYd5+GfAH4KfAV0hLYrSltWSxnmNvZlYj3Q5OEfGGpEXAkcB9wDxgFGnNpIXAnRFxaPEYSUOBBRGxRxunvQ8YJeknEfF66VJtlD0fOCcibsrrO43P9Vos6TlJe5OC2+FtHA+wBNgSWCJpTWBD0jIa1sA8y86scVVqQsQUUkbxKcBU4BhSEtYZwJ6SBgNIWl/S9sCjwEBJe+Tta0nasXC+XwG3AtflYDETGClp4zwkeFCh7IbAX/PrI1rU61LS8N5vI+Ktdup/U+HYMcDd4aeTzcxqplLBaSqwGTA9Ip4DXgem5mG4scA1kuaRgtWQiPgXKQicLWkuKZD9W/GEEXEOaYjwKuA5Uo9oOnBX3l4ynhTEpgIt7yndBPSj/SE9SMFwY0lPkFIYjSu34WZmVnm9On2RpCbg3Iio+PiO0xeZmXVeuemL6u4h3EqRNA74Gu3fazIzszpUd8HXIv5MAAAPLklEQVRJ0iDgjxGxU5nlL8/lr89Zxs+JiL9ExFmkZd3fzihBmhF4UItTXFeYFWi9wE8O3r+i5/vWtX+s6PnMrGN1F5y6IyL+o4P9Z7L681RmZlaH6jV90RqSLpG0QNIdktaTNEzSDEnzJN0o6T0tD5I0Od9n6umMEmZm1oPqNThtB1wQETsCy4EvkBKznhwRO5MyT3ynrYN7KqOE0xeZmVVHvQanhRExJ7+eTXqgd0BE3JO3XQF8rJ3j384okaetX1vYtwVwu6Rm4ESg9HzVZcCX8+tWM0o4fZGZWXXUa3BaUXj9FjCgC+doL6PEzyNiKPCfwLqQMkoAxYwSf+rCNc3MrAIaZULES8CLkkZExFTgS8A97ZSfCZwnaWPgZdIMvbl5XzkZJa7qIKOE1THPrjNrfI0SnCAFkoskrQ88Rcrl16qIeFbSeFJGiWdJGSXWyLvHkzJK/JWUsaK4Yu5NpOG8jjJKmJlZD+rVGSI6qzMZJZwhwsys8/p8hojOckYJM7P6Ua8TIqouIs6KiK0j4t5a18XMrK9zz8ka1gXH3F2V63z9or2rch0zW6XX9JwkDZB0bOH9SEllT9uSNFXSnPzzjKTf90xNzcysI70mOJGehTq2w1JtiIgRETEsIoaRZvndULGamZlZp9RVcJI0SNIjki6VNF/SREmjJU3Lue92kzRe0mU5j95Tko7Ph58FbJt7PhPytn6Srs/nnChJZdShP7A38I6ek9MXmZlVR10Fp2wwcB6wMzAEOAzYi7QM/Cm5zBBgP2A34Dt56fZxwJO593NiLrcL8E1Sbr1tKCSAbcfngUkR8XLLHU5fZGZWHfUYnBZGRHNOxrqAFCiClOx1UC5zS0SsyIlZnwc2beNc90fEknyuOYXj23MocE13GmBmZt1Tj7P1inn1Vhber2RVfVvm3murHeWWAyCnO9qN1HuyOudZdGa9Vz32nLrqFaB/N89xEGlV3dcrUB8zM+uiXhOcImIZMC1PpJjQ4QGtOwQP6ZmZ1Zxz63WRc+uZmXVeubn1ek3PyczMeo96nBDRoyTdyOrLZACcTFpZ98vAeyKiX9UrVoceHvKhWlehLnzokYdrXQWzPqfPBaeIaHUmnqSXgJ8Dj1e3RmZm1lLVhvUkbSDpFklz86SFgyUNl3SPpNmSbpe0WS77VUkP5LK/ywsMIumgfOxcSVPytnUl/VpSs6SHJI3K28dKukHSbTm7xI/aq19EzIiIZ3v6czAzs45V857TJ4FnIuLDEbETcBtwPjAmIoYDlwFn5rI3RMSuEfFh4GHgqLz9dGC/vP2AvO3rABExlPQA7RWS1s37hgEHA0OBgyVt2Z0GOH2RmVl1VDM4NQOjJZ0taQSwJbATcKekOcBpwBa57E45S3gzafG/HfP2acDlkr7KqmXX9wKuAoiIR4Cnge3zvkkR8VJ+bukvwNbdaYDTF5mZVUfV7jlFxGOShgOfBn4I3AksiIg9Wil+OfC5iJgraSwwMp/jGEm7A58B5kgaBrSXzLVTGSJsdZ4IYGa1Us17TpsD/4yIq4EfA7sDAyXtkfevJanUQ+oPPJsTuh5eOMe2ETEzIk4HXiD1vqaUykjaHtgKeLRKzTIzsx5QzZ7EUGCCpJXAG8DXgDeBn0naMNflp6Rkr/8DzCQN0TWzKi3RBEnbkXpLk4C5wCPARXkI8E1gbESsKGN1jNXkCROHAetLWgJcGhHju95cMzPrKmeI6CJniDAz6zxniDAzs4ZVzXtOt0oa0InygyTN74F6zMyr5RZ/hrYo82qlr2tmZuWr5my9T1frWu2JiN1rXYfuGnrF0I4LWcU0H9Fc6yqY9TkV6zlJOknS8fn1uZLuzq/3kXS1pEWSNsk9ooclXSJpgaQ7JK2Xyw7P2R+mkx+uzdt3lHR/7uXMk7RdPs8jkq7I264vZJJoK/PEtjljxOz8HNWQvP0DkqbnrBRnVOozMTOzrqnksN4UYER+3QT0y1PB9wKmtii7HXBBROwILAe+kLf/Gji+lWefjgHOi4hh+dxL8vYPAhdHxM7Ay8Cx+ZptZZ64GPhG3v5t4MK8/TzgFxGxK/C3rn4AZmZWGZUMTrOB4ZL6kx5+nU4KJCN4Z3BaGBFzCscNytPJB0TEPXn7VYXy04FTJJ0MbB0Rr+XtiyNiWn59NSkQfpBWMk9I6gf8G3Bd3v5LYLN87J6sWmSweN3VOH2RmVl1VOyeU0S8IWkRcCRwHzAPGAVsS8qPV9Qyc8N6pGeXWp3XHhH/K2kmKTPE7ZL+A3iqlfKRz/OOzBOS3g0sz72vVi/TbgNTPS4m9b5oamryHHwzsx5S6QkRU0jDZV8hPTx7DjA7IqKjh2IjYrmklyTtFRH3snpmiG2ApyLiZ/n1zqTgtJWkPSJiOinp672k7BADS9vzMN/2EbFA0kJJB0XEdUoV2jki5pJy9h1C6n0dTp3zDXoz6+0qPZV8KmmobHpEPAe8zjuH9NpzJHBBnhDxWmH7wcD8PBw3BLgyb38YOELSPGAj0n2jfwFjgLMlzQXmkIbzIAWeo/L2BcCBeft/AV+X9ACwYWcabGZmldewGSIkDQL+mJffqDpniDAz6zxniDAzs4bVsEtIRMQi0qw8MzPrZdxzMjOzulP1nlNePPCOiHgmv18ENEXECxW+zq2kJTAADouIC9spuzVwA2l13bWA8yPiokrWp9vGe55GzYx/qdY1MOtzatFzGgtsXokTSWozuEbEpyNiOTAAOLaDUz0L/Ft+Bmp3YFxeHNHMzGqgw+BURs68fXNeugclXZczMSDp9Jyrbr6ki5WMIWWNmJjz5K2XL/ONfHxzId/dBpIuy+d4SNKBefvYfJ2bgTskbSZpSj7ffEkjcrlFkjYBzgK2zfsntNbGiPhXRJQeDF6nrc/FGSLMzKqjnJ5TeznzmknpgUZHxEeAWcAJuezPI2LXPNV7PWD/iLg+lzk8IoYV0hC9kI//BekhXoBTgbtzvrtRpFVwN8j79gCOiIi9SUN3t+dez4dJzzUVjQOezNc7sa1GStoyPy+1GDi7NOxYFBEXR0RTRDQNHDiwg4/NzMy6qpzg1F7OvNeAHYBp+QHZI4Ct83GjlNZOagb2BnZs5xo3FK41KL/elzS8NgeYDKwLbJX33RkRf8+vHwCOlDQeGBoRr5TRpneIiMU5gexg0oO9m3blPGZm1n0dTojoIGfeQlKgOLR4jKR1SRm/myJicQ4c67ZzmdKQ2luFOgn4QkQ82uLcuwP/KNRviqSPkfLuXSVpQkRcSRdFxDOSFpCC7/VdPU/F+aa8mfUh5U6IKOXMm0JKR3QMafhsBrCnpMEAktaXtD2rAtEL+R7UmMK5XgH6l3HN20n3opTPvUtrhfJMu+cj4hLgV8BHWhTp8HqSttCqNaXeQ8pS/mh7x5iZWc8pNzi1mjMvIpaSZt9dk+/XzACG5Flyl5DuSf2eNPRWcjlwUYsJEa05gzSte57Scu1tLQI4Epgj6SHSulDnFXdGxDLSsOP8tiZEAB8CZuace/cAP44IZ1c1M6uRhs2tV2vOrWdm1nnOrWdmZg2rYXPrtVROlnJJQ3nnSrcrImL3HqyamZl1Uq8JTuXI95HaWgm3qgaNu6XWVbAyLTrrM7Wuglmf09uG9daQdImkBZLukLSepMmSmgAkbZKnxZcyTfxe0s15hdzjJJ2Qs1HMkLRRTVtiZtaH9bbgtB1wQUTsCCwnzd5rz06kDBO7AWcC/4yIXUgPGn+5ZWGnLzIzq47eFpwWRkQpfVEx20Rb/hwRr+Qp8S8BN+ftza0d6/RFZmbV0duC04rC61K2iTdZ1c6WWSqK5VcW3q+kj92PMzOrJ33hC3gRMBy4n9UzVdSUb7KbmbWtt/WcWvNj4GuS7gM2qXVlzMysY84Q0UXOEGFm1nnlZohwcOoiSUuBp9spsglQ0aXn61RfaGdfaCO4nb1NvbZz64jocEaZg1MPkTSrnL8OGl1faGdfaCO4nb1No7ezL9xzMjOzBuPgZGZmdcfBqedcXOsKVElfaGdfaCO4nb1NQ7fT95zMzKzuuOdkZmZ1x8HJzMzqjoNThUjaSNKdkh7P/31PK2WGSZqel/SYJ+ngWtS1O8ppZy53m6Tlkv5Y7Tp2laRPSnpU0hOSxrWyfx1J1+b9M/MClw2njHZ+TNKDkt6UVDcpvzqrjHaeIOkv+f/FSZK2rkU9u6uMdh4jqVnSHEn3StqhFvXstIjwTwV+gB8B4/LrccDZrZTZHtguv94ceBYYUOu6V7qded8+wGdJqxPXvN5ltGsN4ElgG2BtYC6wQ4syxwIX5deHANfWut491M5BwM7AlcCYWte5B9s5Clg/v/5aL/59vrvw+gDgtlrXu5wf95wq50Dgivz6CuBzLQtExGMR8Xh+/QzwPNBoa2902E6AiJgEvFKtSlXAbsATEfFURPwL+A2prUXFtl8P7CNJVaxjJXTYzohYFBHzSNn5G1U57fxzRPwzv50BbFHlOlZCOe18ufB2A6AhZsE5OFXOphHxLED+73vbKyxpN9JfOk9WoW6V1Kl2NpD3A4sL75fkba2WiYg3SWuAbVyV2lVOOe3sDTrbzqOAP/VojXpGWe2U9HVJT5JGPo6vUt26pS8smVExku4C3tfKrlM7eZ7NgKuAIyKi7v46rVQ7G0xrPaCWf2GWU6be9YY2lKPsdkr6ItAEfLxHa9QzympnRFwAXCDpMOA04Iierlh3OTh1QkSMbmufpOckbRYRz+bg83wb5d4N3AKcFhEzeqiq3VKJdjagJcCWhfdbAM+0UWaJpDWBDYG/V6d6FVNOO3uDstopaTTpj66PR8SKlvsbQGd/n78BftGjNaoQD+tVzk2s+mvkCOAPLQtIWhu4EbgyIq6rYt0qqcN2NqgHgO0kfSD/ng4htbWo2PYxwN2R7zI3kHLa2Rt02E5JuwC/BA6IiEb9I6ucdm5XePsZ4PEq1q/raj0jo7f8kO49TCL94icBG+XtTcCl+fUXgTeAOYWfYbWue6Xbmd9PBZYCr5H+utuv1nUvo22fBh4j3Qc8NW/7HunLC2Bd4DrgCdLKytvUus491M5d8+/sH8AyYEGt69xD7bwLeK7w/+JNta5zD7XzPGBBbuOfgR1rXedyfpy+yMzM6o6H9czMrO44OJmZWd1xcDIzs7rj4GRmZnXHwcnMzOqOg5OZmdUdByczM6s7/x+LE4ipBYkEeAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZIAAAEKCAYAAAA4t9PUAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzt3Xl4nXWd9/H3N3ubLmnTdE3btLS0dKEsoQUBpSJYUCkqSBWVC1HUkccZvbi0qLgwM88MszGLyMgIiqgPKIgEBIpYkFEpNqVroEtauqRpm3RJmjR7zvf549wph5A0p03vnCWf13Udzr387vt8fz3lfHrv5u6IiIicqoxEFyAiIqlNQSIiIv2iIBERkX5RkIiISL8oSEREpF8UJCIi0i8KEhER6RcFiYiI9IuCRERE+iUr0QUMhDFjxnhJSUmiyxARSSlr1qw56O5FfbUbFEFSUlJCeXl5ossQEUkpZrYrnnbatSUiIv2iIBERkX5RkIiISL8oSEREpF8UJCIi0i8KEhER6RcFiYiI9IuCREQkDb22+wj3vlhJQ0t76J+lIBERSUOPr6ni3hcryc4M/2deQSIikmY6I86KigMsnjWWvOzM0D9PQSIikmbW7j7CwcZW3j9v/IB8noJERCTNPLdpPzmZGSye1ef9Fk8LBYmISBpxd56r2M8lM8cwPC97QD5TQSIikkYqqo9SdaSZJXMHZrcWKEhERNLKc5v2k5lhvG/OuAH7TAWJiEgaea5iP4umjWZ0fs6AfaaCREQkTVTWNFBZ08iSATpbq4uCREQkTTy3aT8AV85RkIiIyCl4esM+zp86ivEj8wb0c0MNEjNbYmZbzKzSzJb3MD/XzB4N5r9qZiXd5k8xs0Yzuz1m2k4z22hm68xMD2IXEQG2Hmhg8/4GrlkwccA/O7QgMbNM4F7gKmAO8HEzm9Ot2S3AEXefAdwD3N1t/j3Asz2sfrG7n+Pupae5bBGRlPT0+moyDK6eP2HAPzvMLZKFQKW773D3NuARYGm3NkuBh4Lhx4DLzcwAzOxaYAdQEWKNIiIpz90pW1/Nu84YQ9Hw3AH//DCDZBKwJ2a8KpjWYxt37wDqgUIzywe+Dnyvh/U68LyZrTGzW0971SIiKWbT3qPsPNTEhxYM/NYIQFaI67Yepnmcbb4H3OPujcEGSqyL3b3azMYCvzOzze7+8js+PBoytwJMmTLlpIsXEUkVZev3kp1pvH8Ar2aPFeYWSRUwOWa8GKjurY2ZZQEjgcPAIuCfzGwn8DfAN8zsNgB3rw7ea4AniO5Cewd3v9/dS929tKhoYG5cJiIy0CIR5+kN+3j3zCIKhg7cRYixwgyS1cBMM5tmZjnAMqCsW5sy4KZg+DpgpUdd6u4l7l4C/Dvwf939+2aWb2bDAYLdX1cCm0Lsg4hIUluz+wj76lu45pyBP1urS2i7tty9I9iKWAFkAg+6e4WZ3QWUu3sZ8ADwsJlVEt0SWdbHascBTwS7u7KAX7j7c2H1QUQk2T29vprcrAwuP2vg7q3VXZjHSHD3Z4Bnuk37dsxwC3B9H+v4bszwDmDB6a1SRCQ1uTsvvFHDpTOLGJYb6s/5CenKdhGRFLX1QCN765q5/KyxCa1DQSIikqJWbq4BYPEsBYmIiJyCFzfXMGfCiAG/t1Z3ChIRkRRU19RG+a7DCd+tBQoSEZGU9IettUQcFs9WkIiIyCl4cXMNo/NzWFBckOhSFCQiIqmmM+K8tLWWy2YVkZnR052mBpaCREQkxazdfYS6pnbemwS7tUBBIiKSclZuriEzw7h0ZnLcR1BBIiKSYlbtOMS5kwsYOSQ70aUAChIRkZTS0RmhovooCyYn/iB7FwWJiEgK2VbTSGtHhLOLRya6lOMUJCIiKWRjVT0A8ycpSERE5BRs2FvH8NwsSgrzE13KcQoSEZEUsqGqnnmTRpKRBNePdFGQiIikiNaOTt7Yd5SzJyfPbi0IOUjMbImZbTGzSjNb3sP8XDN7NJj/qpmVdJs/xcwazez2eNcpIpKutu5vpL3TOXtS8pyxBSEGiZllAvcCVwFzgI+b2ZxuzW4Bjrj7DOAe4O5u8+8Bnj3JdYqIpKUNe+sAkuqMLQh3i2QhUOnuO9y9DXgEWNqtzVLgoWD4MeByCx7IbmbXAjuAipNcp4hIWtpYVU/B0GyKRw1JdClvE2aQTAL2xIxXBdN6bOPuHUA9UGhm+cDXge+dwjpFRNLShqp65k8aSfDv7aQRZpD01FOPs833gHvcvfEU1hltaHarmZWbWXltbW2fxYqIJLOW9k62HmhIut1aAFkhrrsKmBwzXgxU99KmysyygJHAYWARcJ2Z/RNQAETMrAVYE8c6AXD3+4H7AUpLS3sMGxGRVPH6vqN0RJz5SXagHcINktXATDObBuwFlgGf6NamDLgJeAW4Dljp7g5c2tXAzL4LNLr794Ow6WudIiJpp+uK9kG1ReLuHWZ2G7ACyAQedPcKM7sLKHf3MuAB4GEzqyS6JbLsVNYZVh9ERJLFhqp6xgzLZcLIvESX8g5hbpHg7s8Az3Sb9u2Y4Rbg+j7W8d2+1ikiku42VNVxdnHyHWgHXdkuIpL06pvb2VbTyLlJdOv4WAoSEZEkt6EqeiHiOVMUJCIicgrW7q7DjKR6mFUsBYmISJJbt6eOGUXDGJGXHI/W7U5BIiKSxNydtbuPcG6S7tYCBYmISFLbdaiJI03tnDN5VKJL6ZWCREQkia3bEz3Qri0SERE5JWt3H2FoTiZnjhue6FJ6pSAREUlia/fUsaC4gMwkerRudwoSEZEk1dLeyevVR5P2+pEuChIRkSRVUV1PR8ST9or2LgoSEZEktXZ3cl/R3kVBIiKSpNburqN41BDGDk++O/7GUpCIiCQhd2fNriOcOyV5rx/poiAREUlCuw41sf9oC4umjU50KX1SkIiIJKFVOw4BcOH0wgRX0jcFiYhIElq14xBjhuVyRlF+okvpU6hBYmZLzGyLmVWa2fIe5uea2aPB/FfNrCSYvtDM1gWv9Wb24ZhldprZxmBeeZj1i4gkgrvzyo5DXDh9dFI+EbG70B61a2aZwL3AFUAVsNrMytz99ZhmtwBH3H2GmS0D7gZuADYBpcEz2icA683sKXfvCJZb7O4Hw6pdRCSRdh5q4sDRVi46I/l3a0G4WyQLgUp33+HubcAjwNJubZYCDwXDjwGXm5m5e1NMaOQBHmKdIiJJJZWOj0C4QTIJ2BMzXhVM67FNEBz1QCGAmS0yswpgI/CFmGBx4HkzW2Nmt4ZYv4hIQqzacYii4blMH5P8x0cgxF1bQE879rpvWfTaxt1fBeaa2VnAQ2b2rLu3ABe7e7WZjQV+Z2ab3f3ld3x4NGRuBZgyZUp/+iEiMmDcnVU7DnHh9MKUOD4C4W6RVAGTY8aLgere2phZFjASOBzbwN3fAI4B84Lx6uC9BniC6C60d3D3+9291N1Li4qK+t0ZEZGB0HV85MLpyX/9SJcwg2Q1MNPMpplZDrAMKOvWpgy4KRi+Dljp7h4skwVgZlOBWcBOM8s3s+HB9HzgSqIH5kVE0sIr21Pr+AiEuGsrOOPqNmAFkAk86O4VZnYXUO7uZcADwMNmVkl0S2RZsPglwHIzawciwF+5+0Ezmw48EWzuZQG/cPfnwuqDiMhAS7XjIxDuMRLc/RngmW7Tvh0z3AJc38NyDwMP9zB9B7Dg9FcqIpJ4b10/kjrHR0BXtouIJI3X9x2ltqGVd88ck+hSToqCREQkSby0pRaA98xKrROEFCQiIknipS01zJs0IumfP9KdgkREJAnUN7WzZtcRFs8am+hSTpqCREQkCby8rZaIw2UKEhERORUvbamlYGg250xO7uez90RBIiKSYJGI84etNbx7ZhGZGalz2m8XBYmISIJtqq7nYGMbi2en1tlaXRQkIiIJ9uLmWszg3TMVJCIicgpe2lrD2cUFFA7LTXQpp0RBIiKSQDUNLazbU8fiFLsIMZaCREQkgVZs2o87XD1/QqJLOWUKEhGRBPrtxn3MGDuMM8cNT3Qpp0xBIiKSILUNrfzlzcNcPW98okvpFwWJiEiCrKjYT8Th6rNTd7cWKEhERBLmmY37mF6Uz6wU3q0FIQeJmS0xsy1mVmlmy3uYn2tmjwbzXzWzkmD6QjNbF7zWm9mH412niEgqONjYyqodh/jA/Akp9RCrnoQWJGaWCdwLXAXMAT5uZnO6NbsFOOLuM4B7gLuD6ZuAUnc/B1gC/NDMsuJcp4hI0uvarXXVvNTerQXhbpEsBCrdfYe7twGPAEu7tVkKPBQMPwZcbmbm7k3u3hFMzwP8JNYpIpL0nt24n2lj8jlrQmrv1oJwg2QSsCdmvCqY1mObIDjqgUIAM1tkZhXARuALwfx41ikiktRqG1p5Zcchrp4/PuV3a8FJBImZXWJmNwfDRWY2ra9Fepjm8bZx91fdfS5wAXCHmeXFuc6uem81s3IzK6+tre2jVBGRgfObtXvpjDgfPjc9/h0cV5CY2XeArwN3BJOygZ/1sVgVMDlmvBio7q2NmWUBI4HDsQ3c/Q3gGDAvznV2LXe/u5e6e2lRUereekBE0ou788vyPZw7pYAZY1N/txbEv0XyYeAaoj/ouHs10NefwGpgpplNM7McYBlQ1q1NGXBTMHwdsNLdPVgmC8DMpgKzgJ1xrlNEJGmtr6pnW00j158/ue/GKSIrznZtwQ+8A5hZfl8LuHuHmd0GrAAygQfdvcLM7gLK3b0MeAB42MwqiW6JLAsWvwRYbmbtQAT4K3c/GHz2O9YZb2dFRBLtl+V7yMvO4IMLUv9srS7xBskvzeyHQIGZfQ74DPA/fS3k7s8Az3Sb9u2Y4Rbg+h6Wexh4ON51ioikgua2Tp5aV81V8yYwIi870eWcNnEFibv/i5ldARwlupvp2+7+u1ArExFJMysq9tPQ2sH1pcWJLuW0iitIgl1ZK939d2Y2C5hlZtnu3h5ueSIi6eNXa/YwefQQLpxWmOhSTqt4D7a/DOSa2STgBeBm4CdhFSUikm52H2riT5WHuO68yWRkpP61I7HiDRJz9ybgI8B/ufuHid6iRERE4vCTP+8kK8O44YL0OVurS9xBYmYXATcCvw2mxXugXkRkUGtoaeeX5Xv4wNkTGD8yL9HlnHbxBslfA8uBXwen8E4DVoZXlohI+vhVeRWNrR185uK+bgiSmuLdqmgiej3Hx83sk0RvVdLjrUlEROQtnRHnJ3/eSenUUSyYXJDockIRb5D8HLid6O3dI+GVIyKSXn7/xgF2H25i+VWzE11KaOINklp3fyrUSkRE0tADf3yTSQVDuHLOuESXEpp4g+Q7ZvYj4PdAa9dEd/91KFWJiKSBTXvrefXNw3zj6tlkZabvk83jDZKbgdlE7/rbtWvLAQWJiEgvvr+ykuF5WSxbOCXRpYQq3iBZ4O7zQ61ERCSNbD3QwHMV+/nye2ek1X21ehLvttYqPRtdRCR+P3ixkqE5mdycpqf8xop3i+QS4CYze5PoMRID3N3PDq0yEZEUtfPgMcrWV/PZS6czKj8n0eWELt4gWRJqFSIiaeS+l7aTlZnBZy9N/60RiP828rvCLkREJB1UHWni8dequHHRFMYOT7/bofQkfc9HExFJgP/8/TYyzPj8e85IdCkDJtQgMbMlZrbFzCrNbHkP83PN7NFg/qtmVhJMv8LM1pjZxuD9vTHLvBSsc13wGhtmH0RE4lVZ08Bja6r41EVTmVgwJNHlDJjQ7uBrZpnAvcAVQBWw2szK3P31mGa3AEfcfYaZLQPuBm4ADgIfcvdqM5tH9Bntk2KWu9Hdy8OqXUTkVPzLiq0Mzcniry4bPFsjEO4WyUKg0t13uHsb8AiwtFubpcBDwfBjwOVmZu6+1t2rg+kVQJ6Z5YZYq4hIv6zbU8dzFfv53KXTKRw2uH6uwgySScCemPEq3r5V8bY27t4B1APdn0H5UWCtu7fGTPtxsFvrTjNLr0eNiUjKcXfufnYzhfk53DJIztSKFWaQ9PQD3/3W8ydsY2Zzie7u+nzM/BuDq+wvDV6f6vHDzW41s3IzK6+trT2pwkVETsbL2w7yyo5DfGnxDIblDr5n/oUZJFVA7DMli4Hq3tqYWRYwEjgcjBcDTwCfdvftXQu4+97gvQH4BdFdaO/g7ve7e6m7lxYVFZ2WDomIdNfa0cn3yiqYWjiUGy9M73tq9SbMIFkNzDSzaWaWAywDyrq1KQNuCoavA1a6u5tZAdFH+t7h7n/qamxmWWY2JhjOBj5I9BkpIiIJ8cAf32THwWN875q55GZlJrqchAgtSIJjHrcRPePqDeCXwWN67zKza4JmDwCFZlYJfJXo43wJlpsB3NntNN9cYIWZbQDWAXuB/wmrDyIiJ7K3rpn/+n0l7587jstmDd4rEcw9/Z+YW1pa6uXlOltYRE6vL/5sDS9uqeGFr76H4lFDE13OaWdma9y9tK92urJdROQUvLilhmc37ee2xTPSMkROhoJEROQk1Te1s/zxDcwYO4zPvXt6ostJOAWJiMhJuvPJTRxqbOOej50zaA+wx1KQiIichKfWV1O2vpovXz6T+cUjE11OUlCQiIjE6cDRFr71m00smFww6O6ndSIKEhGROEQizu2/Wk9rRyf/9rEFZGXq57OL/iREROLw4z/v5H+3HeRbH5jDGUXDEl1OUlGQiIj04Y19R7n72c2876yx3LhocN4G5UQUJCIiJ9DS3snfPLKOEUOy+cePno1uOP5Og+82lSIiJ+Ffn9/ClgMN/PjmCxgzyJ4zEi9tkYiI9GL9njoe+OObfGLRFBYP4ntp9UVBIiLSg/bOCF9/fANFw3NZftXsRJeT1LRrS0SkB/e/vIPN+xu4/1PnMyIvO9HlJDVtkYiIdLO9tpH/+P02rp4/nivnjk90OUlPQSIiEiMSce54fCN5WRl890NzE11OSlCQiIjE+Pmru/jLzsN86wNzGDsiL9HlpAQFiYhIYG9dM//47GYumTGG60uLE11Oygg1SMxsiZltMbNKM1vew/xcM3s0mP+qmZUE068wszVmtjF4f2/MMucH0yvN7D9NVweJyGng7nzziY1EHP7hI/N14eFJCC1IzCwTuBe4CpgDfNzM5nRrdgtwxN1nAPcAdwfTDwIfcvf5wE3AwzHL3AfcCswMXkvC6oOIDB5PrN3LS1tq+dqSWUwePbifeHiywtwiWQhUuvsOd28DHgGWdmuzFHgoGH4MuNzMzN3Xunt1ML0CyAu2XiYAI9z9FY8+bP6nwLUh9kFEBoH6pnb+/rdvcO6UAj59UUmiy0k5YQbJJGBPzHhVMK3HNu7eAdQDhd3afBRY6+6tQfuqPtYJgJndamblZlZeW1t7yp0QkfR3zwtbOdLUxt9dO4/MDO3SOllhBklP34afTBszm0t0d9fnT2Kd0Ynu97t7qbuXFhUVxVGuiAxGm/cf5eFVu/jEoinMnagnHp6KMIOkCpgcM14MVPfWxsyygJHA4WC8GHgC+LS7b49pH3sqRU/rFBGJi7vznScrGJGXxe1Xzkp0OSkrzCBZDcw0s2lmlgMsA8q6tSkjejAd4Dpgpbu7mRUAvwXucPc/dTV2931Ag5ldGJyt9WngyRD7ICJp7KkN+3j1zcPc/v5ZFAzNSXQ5KSu0IAmOedwGrADeAH7p7hVmdpeZXRM0ewAoNLNK4KtA1ynCtwEzgDvNbF3w6rr15heBHwGVwHbg2bD6ICLpq66pjb97+nXmTRrBsgv0sKr+sOjJT+mttLTUy8vLE12GiCSRrzy6jqfWV/ObL13MvEk6NtITM1vj7qV9tdOV7SIy6Kyo2M8Ta/fypcUzFCKngYJERAaVw8fa+OYTG5kzYQRfWjwj0eWkBT2PREQGlW8/uYn65nZ++plF5GTp39Kng/4URWTQWFGxn6c37OP/vHcmcyaOSHQ5aUNBIiKDQn1zO3f+ZhOzxw/ni5edkehy0op2bYnIoPAPz7zBwcZWHrjpArIz9W/o00l/miKS9v5UeZBHVu/hc++ezvxinaV1uilIRCStNbS0s/zXG5g2Jp+vvO/MRJeTlrRrS0TSVvRhVZuormvh0VsvJC87M9ElpSVtkYhI2vrVmirK1lfzlffNpLRkdKLLSVsKEhFJS5U1DXznyQouml7IFy/ThYdhUpCISNppauvgtl+sZUhOJv++7Bw9rCpkChIRSSuRiPPVR9ez9UAD//axBYwbkZfoktKegkRE0sq/PL+F5yr2842rz+KyWWP7XkD6TUEiImnj8TVV/OCl7Xx84RRuuWRaossZNBQkIpIWXtpSw/Jfb+BdZxRy19K5RB+iKgMh1CAxsyVmtsXMKs1seQ/zc83s0WD+q2ZWEkwvNLMXzazRzL7fbZmXgnV2f3KiiAxSf648yOcfXsPMscO578bzdQuUARbaBYlmlgncC1wBVAGrzazM3V+PaXYLcMTdZ5jZMuBu4AagBbgTmBe8urvR3fXIQxFh9c7D3PJQOVMLh/Kzzy5i5NDsRJc06IQZ2wuBSnff4e5twCPA0m5tlgIPBcOPAZebmbn7MXf/I9FAERHp0SvbD3Hzj1czoSCPn3/2Qkbn5yS6pEEpzCCZBOyJGa8KpvXYxt07gHqgMI51/zjYrXWnaUeoyKD02w37uOnBvzBhZB6/+OyFFA3PTXRJg1aYQdLTD7yfQpvubnT3+cClwetTPX642a1mVm5m5bW1tX0WKyKp4yd/epPb/t9rLJg8kl994SLGj9S1IokUZpBUAZNjxouB6t7amFkWMBI4fKKVuvve4L0B+AXRXWg9tbvf3UvdvbSoqOiUOiAiyaW1o5Nv/WYj333qda44axwP37KIgqHanZVoYQbJamCmmU0zsxxgGVDWrU0ZcFMwfB2w0t173SIxsywzGxMMZwMfBDad9spFJOlU1zXzsR+u4merdvP590znvk+er7v5JonQztpy9w4zuw1YAWQCD7p7hZndBZS7exnwAPCwmVUS3RJZ1rW8me0ERgA5ZnYtcCWwC1gRhEgm8ALwP2H1QUSSwwuvH+Brj2+grSPCf3/yPJbMm5DokiSGnWADIG2UlpZ6ebnOFhZJNUdb2rnrqdd5bE0Vs8cP594bz+OMomGJLmvQMLM17l7aVzs92EpEko6788IbNXznyU0caGjltsUz+PLlM8nJ0oWGyUhBIiJJZXttI3c99Tp/2FrLzLHDePyT53PO5IJElyUnoCARkaRw4GgL9720nZ+/uou8rEzu/OAcPn3RVN3uJAUoSEQkoQ4cbeH+l3fws1W76Ig4Hyst5qtXzNIFhilEQSIiCbGxqp4H//QmT2+opjPifOS8Yr783plMKRya6NLkJClIRGTAHG1p5+n1+/hl+R7W7akjPyeTGxdN5eaLS5hamJ/o8uQUKUhEJFSRiLPqzUP8qryKZzfto6U9wpnjhnHnB+dwfWkxI/J0t95UpyARkVDsOnSM36yt5rHX9rDncDPD87L46HnFfKx0MmcXj9SDp9KIgkRETpu9dc08X7GfsvXVrN1dhxlcNL2Q26+cxfvnjtctTdKUgkRETll7Z4QNVXX877aDvPDGATbtPQrA7PHDueOq2VxzzkQmjByS4ColbAoSEYlbW0eETdX1rH7zMKt3HmbVjsM0tnZgBudOLuCOq2ZzxZxxTNdtTAYVBYmI9CgScbbXNrJ2dx0b99azcW89b+w7SmtHBICSwqF8aMFELp05houmFzJKTycctBQkIgLA4WNtrNtzhHW761i7p451u+toaO0AYFhuFnMnjuBTF07lvKmjKC0ZxdjhepiURClIRAaZoy3tbK9pZHvtMbbVNLBlfwOb9zWw/2gLABkGZ44bzofOmci5kws4d8oopo/JJyNDZ1lJzxQkImmmvTNCTUMr1XXNVB1poupwM7sON7Hz4DF2HjrGwca2421zMjM4Y+wwLjqjkNnjh7NgcgHzJ40kP1c/DRI//W0RSSHuzqFjbVQdaWZ/fTPVdS3sq2+mur6F6rpmquuaqW1oJdLtMUNjh+dSMiafy2ePo2RMPjPGDmPG2GFMHjWELN0UUfpJQSKShNydA0db2VbTwNYDjWw70MC2mkYqaxqpb25/W9ucrAwmFQxhYkEel84sYuLIPCYUDGFiwRAmj4q+6/oNCVOoQWJmS4D/IPpY3B+5+z92m58L/BQ4HzgE3ODuO82sEHgMuAD4ibvfFrPM+cBPgCHAM8Bfn+g57yLJKhJxDjS0sOdwMzsPHuPNQ8ei7wePsetQE83tncfbjs7PYcbYYXzg7AnMKBrG1MKhjB+Zx4SRQxg1NFtXiUtChRYkZpYJ3AtcAVQBq82szN1fj2l2C3DE3WeY2TLgbuAGoAW4E5gXvGLdB9wKrCIaJEuAZ8Pqh0h/1Te3s3nfUbYcaODNg9Gw2HWoiaojzbR1Ro63y8owpoweSsmYfN51xhimjRnKGWOHcea44YwZpluqS/IKc4tkIVDp7jsAzOwRYCkQGyRLge8Gw48B3zczc/djwB/NbEbsCs1sAjDC3V8Jxn8KXIuCRJJAfXM7lTWNbK9pZFtNdFfUtgON7K1rPt5maE4mJYX5zJ4wnCvmjqN41FAmjxrCtDH5TCrQ8QpJTWEGySRgT8x4FbCotzbu3mFm9UAhcPAE66zqts5JPTU0s1uJbrkwZcqUk61dpEedEWdffTM7Dzbx5sHG48ctttU0UtvQerxdblYGZxQNo7RkFJ8cP5XZE4Zz1vgRjBuRq91QknbCDJKe/m/pfiwjnjan1N7d7wfuBygtLdUxFIlLZ8Q51NhKVV0zew5Hdz9VHYm+7z3S/I7dUcNys5gxdhjvObMoeiZUUXA21OihZOq6CxkkwgySKmByzHgxUN1LmyozywJGAof7WGdxH+sUOc7dae2IUNfUTl1zG3VN7Rw+1sahY20cbmzjYGMrNQ0t1Da0cuBoKweOttDR7dzZwvwcikcP5ayJI7hi7jimFeYztTCfaWPytYUhQrhBshqYaWbTgL3AMuAT3dqUATcBrwDXAStPdAaWu+8zswYzuxB4Ffg08F9hFA9QUV1PS3sEM8gwI+P4u5GZYZhFN5EsmGcxbayX94yYtt3X2zWeGQyn8w9UJOK0RyK0dzqwkMFkAAAIxElEQVQdnRHaOiK0xby3tkffW9o7aW2P0NLRSUt7dLzr1dzeSVNbJ81tnRxr66SptYPG4HWstYOGlg6OtrTT3tn7BmnB0GzGDs9lzLBcFk0bzYSCPMaPHMLEkXlMHj2U4lFDGJqjs+RFTiS0/0OCYx63ASuInv77oLtXmNldQLm7lwEPAA+bWSXRLZFlXcub2U5gBJBjZtcCVwZnfH2Rt07/fZYQD7T/9SPrqKxpDGv1fTKDzCB8CEKrK3C6ho9Pz7Djofb2cAuW4a3wii729iA0OL4u6DnEujLej/8n+ubuwTs4TiQSnRZx6HSnMxJ9dXRGaA/eu18wdyoyM4wh2ZkMzckkPzeLIdmZDMvLYvyIPPJzsxiel8WIIdkMz8uiYEgOBUOzGTkkm9H5ORTm5zAqP4dsHdwW6TcbDJdglJaWenl5+UkvV74zeovs4z+WHt2HHnGnMxL90XSHSPBnGPHgR/T48Fs/sJ3u0PXjenx6dF3Rdby1TNcPcNf8rs8iaBP9wX5rGGKnv7U+gh/1SOwPvXsPP/5vjRNdjLcGHYs9NGVvvXWFTXQ4duvsrRDLzMggMyMaiFmZGWRlGtkZGWRmGDlZGWRnGlkZGeRkZZCTmUF2lpGblUluVnRaXnZ0OC87k7ysTPJyosNDsjMVAiIhM7M17l7aVztts59AacnoRJcgIpL09E86ERHpFwWJiIj0i4JERET6RUEiIiL9oiAREZF+UZCIiEi/KEhERKRfFCQiItIvg+LKdjOrBXYluo4YY+j9VvmpJl36ki79APUlWaViX6a6e1FfjQZFkCQbMyuP57YDqSBd+pIu/QD1JVmlU1+6064tERHpFwWJiIj0i4IkMe5PdAGnUbr0JV36AepLskqnvryNjpGIiEi/aItERET6RUEygMzsdjNzMxvTy/ybzGxb8LppoOuLh5n9rZltMLN1Zva8mU3spV1n0GadmZUNdJ19OYl+pMJ38s9mtjnozxNmVtBLu51mtjHo88k/6W0AnERflpjZFjOrNLPlA11nX8zsejOrMLOImfV6plYqfCfx0K6tAWJmk4EfAbOB8939YLf5o4FyoJToMwrXBO2ODHStJ2JmI9z9aDD8ZWCOu3+hh3aN7j5swAuMUzz9SKHv5EpgZfB467sB3P3rPbTbCZR2/7uXTOLpi5llAluBK4AqYDXw8eBR3EnBzM4CIsAPgdvdvceQSIXvJB7aIhk49wBf420Psn2b9wO/c/fDwQ/V74AlA1VcvLp+fAP59N6fpBZnP1LlO3ne3TuC0VVAcSLr6Y84+7IQqHT3He7eBjwCLB2oGuPh7m+4+5ZE1zFQFCQDwMyuAfa6+/oTNJsE7IkZrwqmJR0z+3sz2wPcCHy7l2Z5ZlZuZqvM7NoBLC9ucfQjZb6TGJ8Bnu1lngPPm9kaM7t1AGs6Vb31JRW/l96k2nfSIz2z/TQxsxeA8T3M+ibwDeDKvlbRw7SE/Gv/RH1x9yfd/ZvAN83sDuA24Ds9tJ3i7tVmNh1YaWYb3X17iGW/w2noR8p8J0GbbwIdwM97Wc3FwXcyFvidmW1295fDqbh3p6EvSfG9xNOPOCTFd9JfCpLTxN3f19N0M5sPTAPWmxlEN9VfM7OF7r4/pmkVcFnMeDHwUijF9qG3vvTgF8Bv6SFI3L06eN9hZi8B5wIDGiSnoR8p850EJwJ8ELjceznwGfOd1JjZE0R3EQ34j9Zp6EsVMDlmvBioPn0Vxuck/n6daB1J8Z30l3ZthczdN7r7WHcvcfcSov8TnNctRABWAFea2SgzG0V0C2bFAJfbJzObGTN6DbC5hzajzCw3GB4DXAwkzYFQiK8fpM53sgT4OnCNuzf10ibfzIZ3DRPty6aBqzI+8fSF6MH1mWY2zcxygGVA0p0Z2JdU+U7i4u56DeAL2AmMCYZLgR/FzPsMUBm8bk50rb3U/zjRv+wbgKeASd37ArwL2AisD95vSXTdp9KPFPpOKokeM1gXvP47mD4ReCYYnh58H+uBCqK7XxJe+6n0JRi/muiZW9uTsS/Ah4n+o7EVOACsSNXvJJ6XTv8VEZF+0a4tERHpFwWJiIj0i4JERET6RUEiIiL9oiAREZF+UZCInICZNfZz+ceCq/tP1OalE90hNt423doXmdlz8bYX6Q8FiUhIzGwukOnuOwb6s929FthnZhcP9GfL4KMgEYmDRf2zmW0Knh9xQzA9w8x+EDx74mkze8bMrgsWuxF4MmYd9wU3sqwws+/18jmNZvavZvaamf3ezIpiZl9vZn8xs61mdmnQvsTM/jdo/5qZvSum/W+CGkRCpSARic9HgHOABcD7gH82swnB9BJgPvBZ4KKYZS4m+gyTLt9091LgbOA9ZnZ2D5+TD7zm7ucBf+Dt9//KcveFwN/ETK8Brgja3wD8Z0z7cuDSk++qyMnRTRtF4nMJ8P/cvRM4YGZ/AC4Ipv/K3SPAfjN7MWaZCUBtzPjHgluFZwXz5hC9RUusCPBoMPwz4Ncx87qG1xANL4Bs4Ptmdg7QCZwZ076G6C05REKlIBGJT0+3Lj/RdIBmIA/AzKYBtwMXuPsRM/tJ17w+xN7DqDV47+St/3e/QvReTguI7mFoiWmfF9QgEirt2hKJz8vADWaWGRy3eDfwF+CPwEeDYyXjePtt598AZgTDI4BjQH3Q7qpePicD6DrG8olg/ScyEtgXbBF9CsiMmXcmqXo3WUkp2iIRic8TRI9/rCe6lfA1d99vZo8DlxP9wd4KvArUB8v8lmiwvODu681sLdG7vO4A/tTL5xwD5prZmmA9N/RR1w+Ax83seuDFYPkui4MaREKlu/+K9JOZDXP3RjMrJLqVcnEQMkOI/rhfHBxbiWddje4+7DTV9TKw1KPPmxcJjbZIRPrvaTMrAHKAv/XgoWXu3mxm3yH6PPHdA1lQsPvt3xQiMhC0RSIiIv2ig+0iItIvChIREekXBYmIiPSLgkRERPpFQSIiIv2iIBERkX75/1g2MqHJbGoTAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "RMSE on Training set : 0.08677546026255435\n",
      "RMSE on Test set : 0.09048972260089112\n",
      "r2_score on Training set : 0.8428466299996669\n",
      "r2_score on Test set : 0.8545452252922979\n"
     ]
    }
   ],
   "source": [
    "# 3* Lasso\n",
    "#1. 生成学习器实例，LassoCV默认参数可自动确定alpha的搜素范围\n",
    "lasso = LassoCV()\n",
    "\n",
    "#2.模型训练\n",
    "lasso.fit(X_train, y_train)\n",
    "alpha = lasso.alpha_\n",
    "print(\"Best alpha :\" , alpha)\n",
    "\n",
    "#3. 模型性能：cv\n",
    "mse_cv = np.mean(lasso.mse_path_, axis = 1)\n",
    "rmse_cv = np.sqrt(mse_cv)\n",
    "print(\"cv of rmse :\", min(rmse_cv))\n",
    "\n",
    "# 4. 特征重要性\n",
    "#Plot important coefficients\n",
    "coefs = pd.Series(lasso.coef_, index = feat_names)\n",
    "print(\"Lasso picked \" + str(sum(coefs != 0)) + \" features and eliminated the other \" +  \\\n",
    "      str(sum(coefs == 0)) + \" features\")\n",
    "imp_coefs = pd.concat([coefs.sort_values().head(10),\n",
    "                     coefs.sort_values().tail(10)])\n",
    "imp_coefs.plot(kind = \"barh\")\n",
    "plt.title(\"Coefficients in the Lasso Model\")\n",
    "plt.show()\n",
    "\n",
    "#5. 显示不同alpha对应的模型性能\n",
    "plt.plot(np.log10(lasso.alphas_), mse_cv) \n",
    "\n",
    "plt.xlabel('log(alpha)')\n",
    "plt.ylabel('mse')\n",
    "plt.show() \n",
    "\n",
    "#训练误差\n",
    "y_train_pred = lasso.predict(X_train)\n",
    "rmse_train = np.sqrt(mean_squared_error(y_train,y_train_pred))\n",
    "print(\"RMSE on Training set :\" ,rmse_train)\n",
    "\n",
    "#测试误差\n",
    "y_test_pred = lasso.predict(X_test)\n",
    "rmse_test = np.sqrt(mean_squared_error(y_test,y_test_pred))\n",
    "print(\"RMSE on Test set :\" ,rmse_test)\n",
    "\n",
    "r2_score_train = r2_score(y_train,y_train_pred)\n",
    "r2_score_test = r2_score(y_test,y_test_pred)\n",
    "print(\"r2_score on Training set :\" , r2_score_train)\n",
    "print(\"r2_score on Test set :\" , r2_score_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "instant         0.000000\n",
       "season_1       -0.113902\n",
       "season_2       -0.000000\n",
       "season_3        0.000000\n",
       "season_4        0.051772\n",
       "mnth_1         -0.017814\n",
       "mnth_2         -0.005648\n",
       "mnth_3          0.031148\n",
       "mnth_4         -0.000000\n",
       "mnth_5          0.041875\n",
       "mnth_6          0.000000\n",
       "mnth_7         -0.045531\n",
       "mnth_8          0.001251\n",
       "mnth_9          0.073725\n",
       "mnth_10         0.040294\n",
       "mnth_11        -0.024411\n",
       "mnth_12        -0.017943\n",
       "weathersit_1    0.044661\n",
       "weathersit_2    0.000000\n",
       "weathersit_3   -0.164486\n",
       "weekday_0      -0.031634\n",
       "weekday_1      -0.015450\n",
       "weekday_2      -0.002620\n",
       "weekday_3       0.000000\n",
       "weekday_4       0.005422\n",
       "weekday_5       0.008903\n",
       "weekday_6       0.009312\n",
       "temp            0.340128\n",
       "atemp           0.100278\n",
       "hum            -0.186236\n",
       "windspeed      -0.158992\n",
       "holiday        -0.034831\n",
       "workingday      0.000013\n",
       "yr              0.223377\n",
       "dtype: float64"
      ]
     },
     "execution_count": 53,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "coefs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Lasso模型增加了L1正则，系数值进行了收缩，同时有些特征的系数为0。\n",
    "在这个例子中，岭回归模型比Lasso模型性能稍好。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 对测试集进行测试，生成提交文件"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [],
   "source": [
    "y_test_pred = ridge.predict(X_test)\n",
    "\n",
    "#生成提交测试结果\n",
    "df = pd.DataFrame({\"instant\":testID, 'cnt':y_test_pred})\n",
    "df.to_csv('submission.csv')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "Int64Index: 147 entries, 196 to 239\n",
      "Data columns (total 2 columns):\n",
      "instant    147 non-null int64\n",
      "cnt        147 non-null float64\n",
      "dtypes: float64(1), int64(1)\n",
      "memory usage: 3.4 KB\n"
     ]
    }
   ],
   "source": [
    "df.info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "_change_revision": 0,
  "_is_fork": false,
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}
